2012/9/1 Yann Rouillard <span dir="ltr"><<a href="mailto:yann@pleiades.fr.eu.org" target="_blank">yann@pleiades.fr.eu.org</a>></span><br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Maciej,<br><br><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="gmail_quote"><div><div><br></div></div><div>
What are the cases where the new checkpkg code is called on an older structure data ?</div></div></div></blockquote><div><br></div></div><div>This will happen immediately after someone checks out the updated code.</div></div>
</blockquote><div><br></div></div><div>When I recompile a package with my code, it seems to update the data on my package and I don't have errors about that.</div><div>What do I get wrong ?</div></div></blockquote><div>
<br></div><div>For new packages it'll work, but if you want to check a previously indexed package, you'll get a crash. It makes sense to check the whole catalog after adding a new check, because it's useful to analyze already existing packages for issues we might have previously missed.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="gmail_quote">
<div><br></div><div>I can easily ignore the test if needed_symbols is not present. What is your advice ? Is is costly to reindex all packages ?</div></div></div></blockquote><div><br></div></div><div>Costly ‒ yes, but it's a one time operation. I've done it a couple of times already. It can be done in parallel with normal operation; you import into a new database, and then you swap databases together with checking in the code.</div>
</div></blockquote><div><br></div></div><div>Ok, how and when can we do this ?</div></div></blockquote><div><br></div><div>It's been a while since I last did it, so I don't recall all the steps; I had a shell script to do it. It was calling pkgdb to import packages one by one, over catalogs one by one. The operation is IO bound I think, so there isn't much point in making it parallel.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div> </div><div><br></div><div>Before I may want to do some changes in the way the data are stored. Currently I store them in a dictionnary with the soname as the key. The problem is that this information is not always present in binaries. I only know they are at least present if direct binding is enabled.</div>
<div>As I wanted to check if direct binding was enabled, this was not a problem but I wonder if it wouldn't be a better to store the list of symbol as a simple list contains for each symbol the name of the symbol, the soname and version interface providing it if available.</div>
<div><br></div><div>This will take more space in databases (and in rest transfers) but if for some reasons, we need to add some checks which just are interested in symbol presence, they would work whether direct binding is enabled or not.</div>
</div></blockquote><div><br></div><div>Yes, this is a good approach. If you store the data in a way that well resembles the original structure, it'll be easier to make write future checks.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote">
<div><br></div><div>I also wonder if we should use "elfdump -v" rather than "dump -Lv" to extract soname needed as we get in addition the version information of the soname with elfdump. That would allow us to easily add a check to make sure our binary don't work only on the last solaris update.</div>
</div></blockquote><div><br></div><div>Sounds good.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote">
<div><br></div><div>What do you think about all this ?</div></div></blockquote><div><br></div><div>We need to plan the database update. The rough outline of steps to take, from memory:</div><div><br></div><div>- increment the data structure version in the code so that checkpkg knows we're messing with it</div>
<div>- create a new empty database, named checkpkg2 (will be renamed later)</div><div>- alter ~/.checkpkg/... config in your home directory to make checkpkg use the new database (needs to use the relmgr credentials, I'll give them to you)</div>
<div>- stop the catalog generation to avoid race conditions</div><div>- using the new code import the complete catalog into the database</div><div>- submit the code into the repository</div><div>- send a notification to maintainers@ that people should update the code and not use the buildfarm for the next 2h or so (to have some margin)</div>
<div>- move the old checkpkg database out of the way, rename the 'checkpkg2' database to 'checkpkg' (I'm quoting because I mean the literals)</div><div>- now people with the old code can't do anything, will be getting error messages about data mismatches (they shouldn't index new packages and storing the old data structure, there's a safety check to prevent that)</div>
<div>- people with the new code should be able to work normally</div><div>- revert ~/.checkpkg (your homedir only) to the old state (or remove it altogether)</div><div><br></div><div>Maciej</div></div>