The Life of RubenV - Gifgroen!
Performance tip van de dag

Als jouw Firefox, net als de mijne, grote hoeveelheden I/O doet bij het sluiten en traag is bij het gebruiken van de awesomebar, probeer dan het volgende (eerst Firefox volledig sluiten):

for f in ~/.mozilla/firefox/*/*.sqlite; do sqlite3 $f 'VACUUM;'; done


Het is onschadelijk, er zal geen data verloren gaan: het compacteert je SQLite databases. Merkbare verschillen hier.

Tuesday 6 January 2009 14:54
Reactie van Qball
Perfect! een wereld van verschil.
6 January 2009 15:21
Reactie van Chris
Big thanks. Seems to work.

Just noticed: I set history to 10 days about a month ago because awsomebar was soooo slow but it still keeps all my history in the database.
6 January 2009 15:27
Reactie van Aquarion
For OS X, this is:

for f in ~/Library/Application Support/Firefox/Profiles/*/*.sqlite; do sqlite3 "$f" 'VACUUM;'; done
www.aquarionics.com | 6 January 2009 16:19
Reactie van Dav Glass
You can do this on OSX:

cd ~/Library/Application Support/Firefox/Profiles/
for f in */*.sqlite; do sqlite3 $f 'VACUUM;'; done
blog.davglass.com | 6 January 2009 16:19
Reactie van Stoffe
Maybe a bug should be filed to ask Mozilla to set the database(s) to auto-vacuum, or maybe this should be done on shutdown at least.
6 January 2009 16:37
Reactie van Ulisse
Thanks a lot, my awesomebar is really quickier now!
ulisse.wordpress.com | 6 January 2009 16:47
Reactie van michele
Yeah! tip of the year!!! :-)

Finally Firefox starts and stops in reasonable times!

Thanks a lot!
6 January 2009 16:58
Reactie van Stefano
It is also useful for liferea, for sqlite files in ~/.liferea-1.4/mozilla/liferea/ . Thanks!
poisson.phc.unipi.it/~maggiolo | 6 January 2009 17:26
Reactie van Ian McKellar
This is the problem with using SQLite in desktop apps - I need to be my own frickin' DBA!
ian.mckellar.org | 6 January 2009 17:52
Reactie van Paul
Finally - I have a responsive Firefox again. Thanks!
www.pulpmovies.com | 6 January 2009 18:02
Reactie van usar
didn't work for me on Ubuntu 8.10. I get a string of:
Unable to open database "~/.mozilla/firefox/432n677u.default": unable to open database file. Same thing when I sudo su to log in as root and run it.
6 January 2009 18:13
Reactie van RubenV
@usar: Double check that you used the right command, looks like there's a piece of the matching pattern missing.
www.savanne.be | 6 January 2009 18:34
Reactie van Aaron Strontsman
Awesomestest! Thank you.
6 January 2009 19:22
Reactie van mike
SQL error: constraint failed
6 January 2009 20:54
Reactie van Richard Rebel
A general purpose shell gotcha safe version that will do all your sqlite files:

find ~ -name '*.sqlite' -exec sqlite3 '{}' 'VACUUM;' ;

Works on directories containing spaces and is safe fro cronjob use.
6 January 2009 20:59
Reactie van rys
What about windows? How do we get a command line for Mysql lite??
6 January 2009 21:25
Reactie van Artem Vakhitov
How do I do that on Windows? I too want my Firefox not freeze when using the not-so-awesomebar.
6 January 2009 21:49
Reactie van RubenV
@rys, @Artem: Unfortunately, I don't have a windows computer (I'm a pure Linux user), so I can't help you out there. If someone does know how to do it: let us know!
www.savanne.be | 6 January 2009 21:58
Reactie van Kimble
Nice! This helped a lot!
Thanks
6 January 2009 23:23
Reactie van Tom
@Aquarion,Dav Glass: Thnx! Unfortunately you forgot escaping the spaces, so the working commands are:

for f in ~/Library/Application Support/Firefox/Profiles/*/*.sqlite; do sqlite3 "$f" 'VACUUM;'; done

and

cd ~/Library/Application Support/Firefox/Profiles/
for f in */*.sqlite; do sqlite3 $f 'VACUUM;'; done

@Richard Rebel: Nice tip, there are indeed other applications who suffer from this. On OS X Tiger, example number 1 being Apple Mail..
www.t-velop.be | 7 January 2009 00:06
Reactie van RubenV
@Tom: Unfortunately, you forgot quoting as well: Application Support contains a space, which won't work. So substitute it for Application\ Support.
www.savanne.be | 7 January 2009 00:52
Reactie van Delta
I couldn't get
find ~ -name '*.sqlite' -exec sqlite3 '{}' 'VACUUM;' ;

to run, so I had to add a backslash ( '\' ) to the last semicolon ( ';' ):
find ~ -name '*.sqlite' -exec sqlite3 '{}' 'VACUUM;' ;

Nice tip, though; thanks!
7 January 2009 01:05
Reactie van Tom
Bummer, I did copy-paste the wrong snippet! Thnx for correcting my corrections ;-)
www.t-velop.be | 7 January 2009 01:18
Reactie van Peter
@Richard Rebel

The problem is running from a cronjob is that you probably want to make sure the appropriate applications are closed first.

It would also help speed up the new summaries in evolution I think.

Also I believe this is on Mozilla's ToDo list but they had issues with doing it automatically that are yet to be resolved.
7 January 2009 04:47
Reactie van Roshan
Holy Jesus! That was one hell of an improvement! Thanks!
7 January 2009 05:37
Reactie van Derek A. Muenzel, Sr.
Not a complete windows HOWTO but this will get you there:

1) Download the Precompiled Binaries For Windows http://www.sqlite.org/download.html.

2) I extracted the file to C:bin.

3) Change into the directory containing the .sqlite files. Example: cd C:Documents and SettingsrootApplication DateMozillaFirefoxl89ueo7d.default

4) Get a before shot: dir *.sqlite

5) Run the VACUUM: for %f in (*.sqlite) do C:binsqlite3.exe %f "VACUUM;"

6) Get an after shot: dir *sqlite

My results:
BEFORE
11/25/2008 01:39 PM 7,168 content-prefs.sqlite
01/04/2009 08:15 PM 178,176 cookies.sqlite
01/06/2009 11:54 PM 744,448 downloads.sqlite
01/06/2009 11:54 PM 576,512 formhistory.sqlite
01/02/2009 05:14 PM 191,488 kongregate.sqlite
11/03/2008 02:37 PM 2,048 permissions.sqlite
01/06/2009 11:54 PM 50,679,808 places.sqlite
08/06/2008 01:11 PM 2,048 search.sqlite
12/30/2008 05:43 PM 4,096 webappsstore.sqlite

AFTER
01/07/2009 12:38 AM 7,168 content-prefs.sqlite
01/07/2009 12:38 AM 99,328 cookies.sqlite
01/07/2009 12:38 AM 389,120 downloads.sqlite
01/07/2009 12:38 AM 558,080 formhistory.sqlite
01/07/2009 12:38 AM 186,368 kongregate.sqlite
01/07/2009 12:38 AM 2,048 permissions.sqlite
01/07/2009 12:39 AM 42,889,216 places.sqlite
01/07/2009 12:39 AM 2,048 search.sqlite
01/07/2009 12:39 AM 2,048 webappsstore.sqlite
damunzy.blogspot.com | 7 January 2009 06:43
Reactie van jshare
For windows:
* get sqlite3 from here:
* http://www.sqlite.org/sqlite-3_6_7.zip
* http://www.sqlite.org/sqlitedll-3_6_7.zip
* unzip them both into c:windowssystem32
* cd to your profile directory (there may be an easier way to specify this from the commandline, but I don't know cmd globbing that well)
* for %f in (*sqlite) do sqlite3 %f "VACUUM;"
* note: quotes instead of apostrophes
7 January 2009 07:28
Reactie van Lee Willis
On Windows you'll probably have to install sqlite3 (http://www.sqlite.org/download.html)

Then (Assuming that the binary is in your Windows path) you can change to the Mozilla Profile directory

C:Documents and Settings<Your Windows Username>Application DataMozillaFirefoxProfiles<random digits>.default/

and run:

sqlite3 places.sqlite "VACUUM;"

Replacing places.sqlite with the file you want to compact - sorry I'm not familiar enough with Windows scripting to work out how to automatically do it for every file - for me the only large sqlite files were:
- places.sqlite
- urlclassifier2.sqlite
www.snugbaby.co.uk | 7 January 2009 10:18
Reactie van Pacho Ramos
To Richard Rebel:

About adding it to a cron job, Should script check if any firefox instance is running? Would it break "something" if it's run while firefox is running?

Thanks for information
7 January 2009 10:49
Reactie van Jeremy
sqlite has correct locking and database semantics, so you should be able to do this with firefox running.
7 January 2009 13:24
Reactie van Pacho Ramos
OK, thanks for the info :-)
7 January 2009 14:34
Reactie van Bill
There should be an addon that does this once a month or so, as a stopgap before it's added into Firefox (or Gecko) proper.
7 January 2009 16:15
Reactie van SchAmane
WOW, thanks for this tip!
8 January 2009 13:26
Reactie van Marius Scurtescu
@Jeremy

Not sure about sqlite having correct locking and database semantics, but you definitely get an error if Firefox is running: "SQL error: database is locked".

Got that on both Linux and Mac OS X.
marius.scurtescu.com | 9 January 2009 18:51
Reactie van Andrew Conkling
Marius: Jeremy's tip about it running this command with Firefox running still is sorta valid; it won't optimize Firefox's databases, but it won't harm them either.

So if you're setting up a cron job, it will do its thing on whichever databases are available. It's not the kind of thing you'd need to run every day, so just make sure you have Firefox (or your other apps) closed at least sometimes. ;)

Also, this wouldn't pick up any sqlite databases that don't end in ".sqlite", so there could be further gains to be made here. For example, I ran this on Banshee's database (~/.config/banshee-1/banshee.db) and found a decent performance gain in the application.
andrewski.net | 30 January 2009 02:14
Reactie toevoegen:
Naam
Email (optioneel)
Website (optioneel)
Onthoud mijn gegevens
Email mij wanneer er nieuwe reacties zijn
Reactie
You are here:
I speak:
More:
The past: