[csw-devel] SF.net SVN: gar:[20932] csw/mgar/gar/v2/lib/web/releases_web.py

[wahwah at users.sourceforge.net]

Revision: 20932
          http://gar.svn.sourceforge.net/gar/?rev=20932&view=rev
Author:   wahwah
Date:     2013-05-02 07:46:58 +0000 (Thu, 02 May 2013)
Log Message:
-----------
releases-web: Do not read whole file into memory

Do it in chunks instead: less RAM usage.

Modified Paths:
--------------
    csw/mgar/gar/v2/lib/web/releases_web.py

Modified: csw/mgar/gar/v2/lib/web/releases_web.py
===================================================================
--- csw/mgar/gar/v2/lib/web/releases_web.py	2013-05-02 07:21:52 UTC (rev 20931)
+++ csw/mgar/gar/v2/lib/web/releases_web.py	2013-05-02 07:46:58 UTC (rev 20932)
@@ -58,9 +58,14 @@
     web.header(
         'Content-type',
         'application/x-vnd.opencsw.pkg;type=upload-results')
-    hash = hashlib.md5()
-    hash.update(x['srv4_file'].value)
-    data_md5_sum = hash.hexdigest()
+    file_hash = hashlib.md5()
+    # Don't read the whole file into memory at once, do it in small chunks.
+    chunk_size = 2 * 1024 * 1024
+    data = x['srv4_file'].file.read(chunk_size)
+    while data:
+      file_hash.update(data)
+      data = x['srv4_file'].file.read(chunk_size)
+    data_md5_sum = file_hash.hexdigest()
     declared_md5_sum = x['md5_sum']
     basename = x['basename']
     save_attempt = False

This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.