The Life of RubenV - Skyblue!
Performance tip of the day

If your Firefox, like mine, insists on doing huge amounts of I/O when closing, as well as acts slow when using the awesomebar, try the following (close down Firefox completely first):

for f in ~/.mozilla/firefox/*/*.sqlite; do sqlite3 $f 'VACUUM;'; done


It's harmless, no data will be lost: it compacts your SQLite databases. Noticable gains for me.

Tuesday January 6 2009 14:54
Comment from Qball
Perfect! een wereld van verschil.
January 6, 2009 15:21
Comment from Chris
Big thanks. Seems to work.

Just noticed: I set history to 10 days about a month ago because awsomebar was soooo slow but it still keeps all my history in the database.
January 6, 2009 15:27
Comment from Aquarion
For OS X, this is:

for f in ~/Library/Application Support/Firefox/Profiles/*/*.sqlite; do sqlite3 "$f" 'VACUUM;'; done
www.aquarionics.com | January 6, 2009 16:19
Comment from Dav Glass
You can do this on OSX:

cd ~/Library/Application Support/Firefox/Profiles/
for f in */*.sqlite; do sqlite3 $f 'VACUUM;'; done
blog.davglass.com | January 6, 2009 16:19
Comment from Stoffe
Maybe a bug should be filed to ask Mozilla to set the database(s) to auto-vacuum, or maybe this should be done on shutdown at least.
January 6, 2009 16:37
Comment from Ulisse
Thanks a lot, my awesomebar is really quickier now!
ulisse.wordpress.com | January 6, 2009 16:47
Comment from michele
Yeah! tip of the year!!! :-)

Finally Firefox starts and stops in reasonable times!

Thanks a lot!
January 6, 2009 16:58
Comment from Stefano
It is also useful for liferea, for sqlite files in ~/.liferea-1.4/mozilla/liferea/ . Thanks!
poisson.phc.unipi.it/~maggiolo | January 6, 2009 17:26
Comment from Ian McKellar
This is the problem with using SQLite in desktop apps - I need to be my own frickin' DBA!
ian.mckellar.org | January 6, 2009 17:52
Comment from Paul
Finally - I have a responsive Firefox again. Thanks!
www.pulpmovies.com | January 6, 2009 18:02
Comment from usar
didn't work for me on Ubuntu 8.10. I get a string of:
Unable to open database "~/.mozilla/firefox/432n677u.default": unable to open database file. Same thing when I sudo su to log in as root and run it.
January 6, 2009 18:13
Comment from RubenV
@usar: Double check that you used the right command, looks like there's a piece of the matching pattern missing.
www.savanne.be | January 6, 2009 18:34
Comment from Aaron Strontsman
Awesomestest! Thank you.
January 6, 2009 19:22
Comment from mike
SQL error: constraint failed
January 6, 2009 20:54
Comment from Richard Rebel
A general purpose shell gotcha safe version that will do all your sqlite files:

find ~ -name '*.sqlite' -exec sqlite3 '{}' 'VACUUM;' ;

Works on directories containing spaces and is safe fro cronjob use.
January 6, 2009 20:59
Comment from rys
What about windows? How do we get a command line for Mysql lite??
January 6, 2009 21:25
Comment from Artem Vakhitov
How do I do that on Windows? I too want my Firefox not freeze when using the not-so-awesomebar.
January 6, 2009 21:49
Comment from RubenV
@rys, @Artem: Unfortunately, I don't have a windows computer (I'm a pure Linux user), so I can't help you out there. If someone does know how to do it: let us know!
www.savanne.be | January 6, 2009 21:58
Comment from Kimble
Nice! This helped a lot!
Thanks
January 6, 2009 23:23
Comment from Tom
@Aquarion,Dav Glass: Thnx! Unfortunately you forgot escaping the spaces, so the working commands are:

for f in ~/Library/Application Support/Firefox/Profiles/*/*.sqlite; do sqlite3 "$f" 'VACUUM;'; done

and

cd ~/Library/Application Support/Firefox/Profiles/
for f in */*.sqlite; do sqlite3 $f 'VACUUM;'; done

@Richard Rebel: Nice tip, there are indeed other applications who suffer from this. On OS X Tiger, example number 1 being Apple Mail..
www.t-velop.be | January 7, 2009 00:06
Comment from RubenV
@Tom: Unfortunately, you forgot quoting as well: Application Support contains a space, which won't work. So substitute it for Application\ Support.
www.savanne.be | January 7, 2009 00:52
Comment from Delta
I couldn't get
find ~ -name '*.sqlite' -exec sqlite3 '{}' 'VACUUM;' ;

to run, so I had to add a backslash ( '\' ) to the last semicolon ( ';' ):
find ~ -name '*.sqlite' -exec sqlite3 '{}' 'VACUUM;' ;

Nice tip, though; thanks!
January 7, 2009 01:05
Comment from Tom
Bummer, I did copy-paste the wrong snippet! Thnx for correcting my corrections ;-)
www.t-velop.be | January 7, 2009 01:18
Comment from Peter
@Richard Rebel

The problem is running from a cronjob is that you probably want to make sure the appropriate applications are closed first.

It would also help speed up the new summaries in evolution I think.

Also I believe this is on Mozilla's ToDo list but they had issues with doing it automatically that are yet to be resolved.
January 7, 2009 04:47
Comment from Roshan
Holy Jesus! That was one hell of an improvement! Thanks!
January 7, 2009 05:37
Comment from Derek A. Muenzel, Sr.
Not a complete windows HOWTO but this will get you there:

1) Download the Precompiled Binaries For Windows http://www.sqlite.org/download.html.

2) I extracted the file to C:bin.

3) Change into the directory containing the .sqlite files. Example: cd C:Documents and SettingsrootApplication DateMozillaFirefoxl89ueo7d.default

4) Get a before shot: dir *.sqlite

5) Run the VACUUM: for %f in (*.sqlite) do C:binsqlite3.exe %f "VACUUM;"

6) Get an after shot: dir *sqlite

My results:
BEFORE
11/25/2008 01:39 PM 7,168 content-prefs.sqlite
01/04/2009 08:15 PM 178,176 cookies.sqlite
01/06/2009 11:54 PM 744,448 downloads.sqlite
01/06/2009 11:54 PM 576,512 formhistory.sqlite
01/02/2009 05:14 PM 191,488 kongregate.sqlite
11/03/2008 02:37 PM 2,048 permissions.sqlite
01/06/2009 11:54 PM 50,679,808 places.sqlite
08/06/2008 01:11 PM 2,048 search.sqlite
12/30/2008 05:43 PM 4,096 webappsstore.sqlite

AFTER
01/07/2009 12:38 AM 7,168 content-prefs.sqlite
01/07/2009 12:38 AM 99,328 cookies.sqlite
01/07/2009 12:38 AM 389,120 downloads.sqlite
01/07/2009 12:38 AM 558,080 formhistory.sqlite
01/07/2009 12:38 AM 186,368 kongregate.sqlite
01/07/2009 12:38 AM 2,048 permissions.sqlite
01/07/2009 12:39 AM 42,889,216 places.sqlite
01/07/2009 12:39 AM 2,048 search.sqlite
01/07/2009 12:39 AM 2,048 webappsstore.sqlite
damunzy.blogspot.com | January 7, 2009 06:43
Comment from jshare
For windows:
* get sqlite3 from here:
* http://www.sqlite.org/sqlite-3_6_7.zip
* http://www.sqlite.org/sqlitedll-3_6_7.zip
* unzip them both into c:windowssystem32
* cd to your profile directory (there may be an easier way to specify this from the commandline, but I don't know cmd globbing that well)
* for %f in (*sqlite) do sqlite3 %f "VACUUM;"
* note: quotes instead of apostrophes
January 7, 2009 07:28
Comment from Lee Willis
On Windows you'll probably have to install sqlite3 (http://www.sqlite.org/download.html)

Then (Assuming that the binary is in your Windows path) you can change to the Mozilla Profile directory

C:Documents and Settings<Your Windows Username>Application DataMozillaFirefoxProfiles<random digits>.default/

and run:

sqlite3 places.sqlite "VACUUM;"

Replacing places.sqlite with the file you want to compact - sorry I'm not familiar enough with Windows scripting to work out how to automatically do it for every file - for me the only large sqlite files were:
- places.sqlite
- urlclassifier2.sqlite
www.snugbaby.co.uk | January 7, 2009 10:18
Comment from Pacho Ramos
To Richard Rebel:

About adding it to a cron job, Should script check if any firefox instance is running? Would it break "something" if it's run while firefox is running?

Thanks for information
January 7, 2009 10:49
Comment from Jeremy
sqlite has correct locking and database semantics, so you should be able to do this with firefox running.
January 7, 2009 13:24
Comment from Pacho Ramos
OK, thanks for the info :-)
January 7, 2009 14:34
Comment from Bill
There should be an addon that does this once a month or so, as a stopgap before it's added into Firefox (or Gecko) proper.
January 7, 2009 16:15
Comment from SchAmane
WOW, thanks for this tip!
January 8, 2009 13:26
Comment from Marius Scurtescu
@Jeremy

Not sure about sqlite having correct locking and database semantics, but you definitely get an error if Firefox is running: "SQL error: database is locked".

Got that on both Linux and Mac OS X.
marius.scurtescu.com | January 9, 2009 18:51
Comment from Andrew Conkling
Marius: Jeremy's tip about it running this command with Firefox running still is sorta valid; it won't optimize Firefox's databases, but it won't harm them either.

So if you're setting up a cron job, it will do its thing on whichever databases are available. It's not the kind of thing you'd need to run every day, so just make sure you have Firefox (or your other apps) closed at least sometimes. ;)

Also, this wouldn't pick up any sqlite databases that don't end in ".sqlite", so there could be further gains to be made here. For example, I ran this on Banshee's database (~/.config/banshee-1/banshee.db) and found a decent performance gain in the application.
andrewski.net | January 30, 2009 02:14
Add a comment:
Name
Email (optional)
Website (optional)
Remember my personal details
Email me when there are new comments
Comment
The author:
RubenV
Ruben Vermeersch
Computer Scientist (Software Engineering), GNOME Hacker, PhD Researcher, Photographer, Earthling
More info | Tweets
You are here:
I speak:
More:
The past: