[csw-devel] SF.net SVN: opencsw:[572] twitter

skayser at users.sourceforge.net skayser at users.sourceforge.net
Mon Oct 24 16:01:34 CEST 2011


Revision: 572
          http://opencsw.svn.sourceforge.net/opencsw/?rev=572&view=rev
Author:   skayser
Date:     2011-10-24 14:01:34 +0000 (Mon, 24 Oct 2011)
Log Message:
-----------
twitter: twitter search to rss, early stage commit

Added Paths:
-----------
    twitter/
    twitter/sweets/
    twitter/sweets/README
    twitter/sweets/sample-search-results.txt
    twitter/sweets/sweets.py


Property changes on: twitter/sweets
___________________________________________________________________
Added: svn:ignore
   + *.pyc


Added: twitter/sweets/README
===================================================================
--- twitter/sweets/README	                        (rev 0)
+++ twitter/sweets/README	2011-10-24 14:01:34 UTC (rev 572)
@@ -0,0 +1,7 @@
+sweets - tools to search tweets and re-publish them through a feed
+
+Twitter has search feeds which have gone from official to unofficial but still
+present. These have caused us troubles in the past, as they contained
+duplicates and ceeswi on #opencsw happily broadcasted redundant items. This is
+a "hacking for fun" attempt to get a sanitized feed version. Early stage, not
+yet usable.

Added: twitter/sweets/sample-search-results.txt
===================================================================
--- twitter/sweets/sample-search-results.txt	                        (rev 0)
+++ twitter/sweets/sample-search-results.txt	2011-10-24 14:01:34 UTC (rev 572)
@@ -0,0 +1 @@
+{"completed_in":0.138,"max_id":127469256991449088,"max_id_str":"127469256991449088","page":1,"query":"opencsw","refresh_url":"?since_id=127469256991449088&q=opencsw","results":[{"created_at":"Fri, 21 Oct 2011 19:40:13 +0000","from_user":"toxickore","from_user_id":402981916,"from_user_id_str":"402981916","geo":null,"id":127469256991449088,"id_str":"127469256991449088","iso_language_code":"es","metadata":{"result_type":"recent"},"profile_image_url":"http://a3.twimg.com/profile_images/1566913558/avatar01_normal.jpg","source":"<a href="http://twitter.com/">web</a>","text":"pft! que basura, el sshd_config se apendeja con el AuthorizedKeysFile http://t.co/PwV8hUgH","to_user_id":null,"to_user_id_str":null},{"created_at":"Fri, 21 Oct 2011 13:16:23 +0000","from_user":"opencsw","from_user_id":83587657,"from_user_id_str":"83587657","geo":null,"id":127372661499236353,"id_str":"127372661499236353","iso_language_code":"en","metadata":{"result_type":"recent"},"profile
 _image_url":"http://a3.twimg.com/profile_images/775037952/opencsw_med_normal.png","source":"<a href="http://twitter.com/download/android" rel="nofollow">Twitter for Android</a>","text":"@esz try pkgutil, it makes upgrades safer, and has additional functionality you might like!","to_user":"esz","to_user_id":29389,"to_user_id_str":"29389"},{"created_at":"Thu, 20 Oct 2011 19:41:34 +0000","from_user":"bonivart","from_user_id":7366785,"from_user_id_str":"7366785","geo":null,"id":127107208226942976,"id_str":"127107208226942976","iso_language_code":"en","metadata":{"result_type":"recent"},"profile_image_url":"http://a1.twimg.com/profile_images/1067284645/0220-0415-RU-1-005_220_normal.jpg","source":"<a href="http://itunes.apple.com/us/app/twitter/id409789998?mt=12" rel="nofollow">Twitter for Mac</a>","text":"@DoctorOctagon: Make a request at http://t.co/azIAV17O. Also, pkgutil -a ccze is eq. to your pipe grep :)","to_us
 er":"DoctorOctagon","to_user_id":1060152,"to_user_id_str":"1060152"},{"created_at":"Tue, 18 Oct 2011 07:31:20 +0000","from_user":"esz","from_user_id":29389,"from_user_id_str":"29389","geo":null,"id":126198664342405120,"id_str":"126198664342405120","iso_language_code":"th","metadata":{"result_type":"recent"},"profile_image_url":"http://a2.twimg.com/profile_images/1543638533/AVT-III_normal.jpg","source":"<a href="http://twitter.com/">web</a>","text":"\u0E16\u0E49\u0E32\u0E04\u0E38\u0E13\u0E43\u0E0A\u0E49 Solaris \u0E08\u0E07\u0E15\u0E34\u0E14\u0E15\u0E31\u0E49\u0E07 pkg-get \u0E41\u0E25\u0E30 OpenCSW \u0E42\u0E14\u0E22\u0E1E\u0E25\u0E31\u0E19 \u0E0A\u0E35\u0E27\u0E34\u0E15 admin \u0E08\u0E30\u0E21\u0E35\u0E04\u0E27\u0E32\u0E21\u0E2A\u0E38\u0E02\u0E02\u0E36\u0E49\u0E19\u0E40\u0E22\u0E2D\u0E30","to_user_id":null,"to_user_id_str":null}],"results_per_page":15,"since_id":0,"since_id_str":"0"}
\ No newline at end of file

Added: twitter/sweets/sweets.py
===================================================================
--- twitter/sweets/sweets.py	                        (rev 0)
+++ twitter/sweets/sweets.py	2011-10-24 14:01:34 UTC (rev 572)
@@ -0,0 +1,68 @@
+#!/usr/bin/env python
+#
+# sweets_search.py:
+#   Search twitter with a search term and store results in a JSON file which
+#   can be used for further processing (e.g. to generate RSS)
+#
+# TODO:
+# * Implement twitter_search (http://search.twitter.com/search.json?q=opencsw)
+# * twitter_search: catch and log twitter API HTTP status codes <> 200
+# * run: write result JSON to output file
+# * get_results: use refresh_url from response for follow up queries
+# * Rewrite file open/close blocks with "with" once python2.5+ on dev box
+#
+
+import json
+import sys
+import time
+import types
+from optparse import OptionParser,SUPPRESS_HELP
+
+def twitter_search_mock(query = "", since = ""):
+    """Search twitter (mockup), returns a single-line, JSON search result."""
+    f = open("sample-search-results.txt", "r")
+    data = ''.join(f.readlines()) # ensure single line result
+    f.close()
+    return data
+twitter_search = twitter_search_mock
+
+def get_results(query):
+    """Trigger twitter search and extract results from JSON response.""" 
+    json_data = twitter_search(query)
+    response = json.read(json_data)
+    return response['results']
+
+def run(interval, query):
+    """Run the main loop which queries Twitter and writes the output."""
+    while 1:
+        results = get_results(query)
+        for result in results:
+            print "%s - http://twitter.com/#!/%s/status/%s" % (
+                    result['text'],
+                    result['from_user'],
+                    result['id_str']
+            )
+        # Tweet URL #http://twitter.com/#!/<from_user>/status/<id_str>
+        if hasattr(interval, "__call__"): interval()
+        else: time.sleep(float(interval))
+
+if __name__ == '__main__':
+    usage = "%prog -s <searchterm> -t <secs>"
+    parser = OptionParser(usage=usage)
+    parser.add_option("-s", dest="searchterm", help="search term to run on Twitter")
+    parser.add_option("-t", dest="interval", default=300,
+                            help="query every X seconds (default 300)")
+    parser.add_option("-d", dest="debug", action="store_true",
+                            help=SUPPRESS_HELP)
+    (options, args) = parser.parse_args()
+
+    required_options = ( "searchterm", )
+    for o in required_options:
+        if not getattr(options, o):
+            parser.print_help()
+            sys.exit(1)
+
+    try:
+        if options.debug: options.interval = sys.stdin.readline
+        run(options.interval, options.searchterm)
+    except KeyboardInterrupt: pass


Property changes on: twitter/sweets/sweets.py
___________________________________________________________________
Added: svn:executable
   + *

This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.



More information about the devel mailing list