SF.net SVN: gar:[23666] csw/mgar/gar/v2/lib/python

wahwah at users.sourceforge.net wahwah at users.sourceforge.net
Tue May 20 00:22:16 CEST 2014


Revision: 23666
          http://sourceforge.net/p/gar/code/23666
Author:   wahwah
Date:     2014-05-19 22:22:15 +0000 (Mon, 19 May 2014)
Log Message:
-----------
checkpkg: Bugfix for disappearing packages

http://lists.opencsw.org/pipermail/maintainers/2014-May/019245.html

Idempotence is hard.

There is a REST URL which allows you to create relational database entries for
a given package. You send a HTTP PUT request, and the package gets registered.
Before that request, the package's metadata only exist as a JSON blob in the
blob table. In order to make any other operations on the package, an entry in
the svr4_file_stats table needs to be created, and this is what this request
does.

What if you send another PUT request at the same URL? REST semantics require
that multiple calls are allowed and that the result is always the same. My
simple-minded solution was to simply nuke all the rows that were created by
this request, and create them anew. The problem is that the function used
deleted too much: it also deleted rows that represent the present of that
package in catalogs. So deleting all the dependent database rows effectively
removed that package from all the catalogs.

The fix is to modify this piece of code to not delete rows that have been
inserted for other causes than just the content of the package.

Modified Paths:
--------------
    csw/mgar/gar/v2/lib/python/models.py
    csw/mgar/gar/v2/lib/python/relational_util.py

Modified: csw/mgar/gar/v2/lib/python/models.py
===================================================================
--- csw/mgar/gar/v2/lib/python/models.py	2014-05-19 22:22:01 UTC (rev 23665)
+++ csw/mgar/gar/v2/lib/python/models.py	2014-05-19 22:22:15 UTC (rev 23666)
@@ -251,9 +251,25 @@
     return str(unicode(self))
 
   def DeleteAllDependentObjects(self):
+    """Prepares the object to be deleted.
+
+    Use this function with caution.
+    """
+    self.DeleteDependentObjectsPopulatedFromPackageItself()
+    logger.debug('Removing all dependent objects from %s; it will cause the '
+                 'package to be removed from all catalogs.', self)
     self.RemoveCatalogAssignments()
+    self.RemoveAllCheckpkgResults()
+
+  def DeleteDependentObjectsPopulatedFromPackageItself(self):
+    """Removing all the objects that only depend on the package contents.
+
+    It doesn't touch rows that are created for other reasons, e.g. assignments
+    of packages to catalogs.
+    """
+    logger.debug('%s - Deleting objects that only depend on the package '
+                 'contents', self)
     self.RemoveAllCswFiles()
-    self.RemoveAllCheckpkgResults()
     self.RemoveOverrides()
     self.RemoveDepends()
     self.RemoveIncompatibles()

Modified: csw/mgar/gar/v2/lib/python/relational_util.py
===================================================================
--- csw/mgar/gar/v2/lib/python/relational_util.py	2014-05-19 22:22:01 UTC (rev 23665)
+++ csw/mgar/gar/v2/lib/python/relational_util.py	2014-05-19 22:22:15 UTC (rev 23666)
@@ -97,11 +97,14 @@
     if "REV" in parsed_basename["revision_info"]:
       rev = parsed_basename["revision_info"]["REV"]
 
-  # If the object already exists in the database, we'll replace all dependent data.
+  # If the object already exists in the database, we'll replace the
+  # object-dependent data, but the gotcha is that we must not delete
+  # assignments of that object to catalogs. If we do so, the package will get
+  # removed from all the catalogs.
   db_pkg_stats = None
   try:
     db_pkg_stats = models.Srv4FileStats.selectBy(md5_sum=md5_sum).getOne()
-    db_pkg_stats.DeleteAllDependentObjects()
+    db_pkg_stats.DeleteDependentObjectsPopulatedFromPackageItself()
   except sqlobject.main.SQLObjectNotFound:
     logger.debug('Package %s not present in the relational db, '
                  'proceeding with insert.', parsed_basename)

This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.



More information about the devel mailing list