[csw-devel] SF.net SVN: opencsw:[572] twitter
skayser at users.sourceforge.net
skayser at users.sourceforge.net
Mon Oct 24 16:01:34 CEST 2011
Revision: 572
http://opencsw.svn.sourceforge.net/opencsw/?rev=572&view=rev
Author: skayser
Date: 2011-10-24 14:01:34 +0000 (Mon, 24 Oct 2011)
Log Message:
-----------
twitter: twitter search to rss, early stage commit
Added Paths:
-----------
twitter/
twitter/sweets/
twitter/sweets/README
twitter/sweets/sample-search-results.txt
twitter/sweets/sweets.py
Property changes on: twitter/sweets
___________________________________________________________________
Added: svn:ignore
+ *.pyc
Added: twitter/sweets/README
===================================================================
--- twitter/sweets/README (rev 0)
+++ twitter/sweets/README 2011-10-24 14:01:34 UTC (rev 572)
@@ -0,0 +1,7 @@
+sweets - tools to search tweets and re-publish them through a feed
+
+Twitter has search feeds which have gone from official to unofficial but still
+present. These have caused us troubles in the past, as they contained
+duplicates and ceeswi on #opencsw happily broadcasted redundant items. This is
+a "hacking for fun" attempt to get a sanitized feed version. Early stage, not
+yet usable.
Added: twitter/sweets/sample-search-results.txt
===================================================================
--- twitter/sweets/sample-search-results.txt (rev 0)
+++ twitter/sweets/sample-search-results.txt 2011-10-24 14:01:34 UTC (rev 572)
@@ -0,0 +1 @@
+{"completed_in":0.138,"max_id":127469256991449088,"max_id_str":"127469256991449088","page":1,"query":"opencsw","refresh_url":"?since_id=127469256991449088&q=opencsw","results":[{"created_at":"Fri, 21 Oct 2011 19:40:13 +0000","from_user":"toxickore","from_user_id":402981916,"from_user_id_str":"402981916","geo":null,"id":127469256991449088,"id_str":"127469256991449088","iso_language_code":"es","metadata":{"result_type":"recent"},"profile_image_url":"http://a3.twimg.com/profile_images/1566913558/avatar01_normal.jpg","source":"<a href="http://twitter.com/">web</a>","text":"pft! que basura, el sshd_config se apendeja con el AuthorizedKeysFile http://t.co/PwV8hUgH","to_user_id":null,"to_user_id_str":null},{"created_at":"Fri, 21 Oct 2011 13:16:23 +0000","from_user":"opencsw","from_user_id":83587657,"from_user_id_str":"83587657","geo":null,"id":127372661499236353,"id_str":"127372661499236353","iso_language_code":"en","metadata":{"result_type":"recent"},"profile
_image_url":"http://a3.twimg.com/profile_images/775037952/opencsw_med_normal.png","source":"<a href="http://twitter.com/download/android" rel="nofollow">Twitter for Android</a>","text":"@esz try pkgutil, it makes upgrades safer, and has additional functionality you might like!","to_user":"esz","to_user_id":29389,"to_user_id_str":"29389"},{"created_at":"Thu, 20 Oct 2011 19:41:34 +0000","from_user":"bonivart","from_user_id":7366785,"from_user_id_str":"7366785","geo":null,"id":127107208226942976,"id_str":"127107208226942976","iso_language_code":"en","metadata":{"result_type":"recent"},"profile_image_url":"http://a1.twimg.com/profile_images/1067284645/0220-0415-RU-1-005_220_normal.jpg","source":"<a href="http://itunes.apple.com/us/app/twitter/id409789998?mt=12" rel="nofollow">Twitter for Mac</a>","text":"@DoctorOctagon: Make a request at http://t.co/azIAV17O. Also, pkgutil -a ccze is eq. to your pipe grep :)","to_us
er":"DoctorOctagon","to_user_id":1060152,"to_user_id_str":"1060152"},{"created_at":"Tue, 18 Oct 2011 07:31:20 +0000","from_user":"esz","from_user_id":29389,"from_user_id_str":"29389","geo":null,"id":126198664342405120,"id_str":"126198664342405120","iso_language_code":"th","metadata":{"result_type":"recent"},"profile_image_url":"http://a2.twimg.com/profile_images/1543638533/AVT-III_normal.jpg","source":"<a href="http://twitter.com/">web</a>","text":"\u0E16\u0E49\u0E32\u0E04\u0E38\u0E13\u0E43\u0E0A\u0E49 Solaris \u0E08\u0E07\u0E15\u0E34\u0E14\u0E15\u0E31\u0E49\u0E07 pkg-get \u0E41\u0E25\u0E30 OpenCSW \u0E42\u0E14\u0E22\u0E1E\u0E25\u0E31\u0E19 \u0E0A\u0E35\u0E27\u0E34\u0E15 admin \u0E08\u0E30\u0E21\u0E35\u0E04\u0E27\u0E32\u0E21\u0E2A\u0E38\u0E02\u0E02\u0E36\u0E49\u0E19\u0E40\u0E22\u0E2D\u0E30","to_user_id":null,"to_user_id_str":null}],"results_per_page":15,"since_id":0,"since_id_str":"0"}
\ No newline at end of file
Added: twitter/sweets/sweets.py
===================================================================
--- twitter/sweets/sweets.py (rev 0)
+++ twitter/sweets/sweets.py 2011-10-24 14:01:34 UTC (rev 572)
@@ -0,0 +1,68 @@
+#!/usr/bin/env python
+#
+# sweets_search.py:
+# Search twitter with a search term and store results in a JSON file which
+# can be used for further processing (e.g. to generate RSS)
+#
+# TODO:
+# * Implement twitter_search (http://search.twitter.com/search.json?q=opencsw)
+# * twitter_search: catch and log twitter API HTTP status codes <> 200
+# * run: write result JSON to output file
+# * get_results: use refresh_url from response for follow up queries
+# * Rewrite file open/close blocks with "with" once python2.5+ on dev box
+#
+
+import json
+import sys
+import time
+import types
+from optparse import OptionParser,SUPPRESS_HELP
+
+def twitter_search_mock(query = "", since = ""):
+ """Search twitter (mockup), returns a single-line, JSON search result."""
+ f = open("sample-search-results.txt", "r")
+ data = ''.join(f.readlines()) # ensure single line result
+ f.close()
+ return data
+twitter_search = twitter_search_mock
+
+def get_results(query):
+ """Trigger twitter search and extract results from JSON response."""
+ json_data = twitter_search(query)
+ response = json.read(json_data)
+ return response['results']
+
+def run(interval, query):
+ """Run the main loop which queries Twitter and writes the output."""
+ while 1:
+ results = get_results(query)
+ for result in results:
+ print "%s - http://twitter.com/#!/%s/status/%s" % (
+ result['text'],
+ result['from_user'],
+ result['id_str']
+ )
+ # Tweet URL #http://twitter.com/#!/<from_user>/status/<id_str>
+ if hasattr(interval, "__call__"): interval()
+ else: time.sleep(float(interval))
+
+if __name__ == '__main__':
+ usage = "%prog -s <searchterm> -t <secs>"
+ parser = OptionParser(usage=usage)
+ parser.add_option("-s", dest="searchterm", help="search term to run on Twitter")
+ parser.add_option("-t", dest="interval", default=300,
+ help="query every X seconds (default 300)")
+ parser.add_option("-d", dest="debug", action="store_true",
+ help=SUPPRESS_HELP)
+ (options, args) = parser.parse_args()
+
+ required_options = ( "searchterm", )
+ for o in required_options:
+ if not getattr(options, o):
+ parser.print_help()
+ sys.exit(1)
+
+ try:
+ if options.debug: options.interval = sys.stdin.readline
+ run(options.interval, options.searchterm)
+ except KeyboardInterrupt: pass
Property changes on: twitter/sweets/sweets.py
___________________________________________________________________
Added: svn:executable
+ *
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
More information about the devel
mailing list