Comments for Jörn's Blog

Comment on Setting up a local DBpedia mirror with Virtuoso by joern

joern — Wed, 01 Feb 2012 14:54:16 +0000

maybe there’s another install of virtuoso interfering with the one you actually want to use? (e.g., one installed by your distribution?)
also see the comment about isql and isql-vt.

Comment on Setting up a local DBpedia mirror with Virtuoso by Nico

Nico — Wed, 01 Feb 2012 14:46:50 +0000

Ho Joern,
thanks for your reply.
Something seems to be going terribly wrong. Obviously, virtuoso is only using a small piece of the available memory, thus leaving everything else to the HDD. Checked with top, which is telling me the same (~250MB in use for virtuoso).
It seems to me like virtuoso is ignoring any settings made in the virtuoso.ini. Looking into the output of the server status, I see the following:

Database Status:
File size 17060331520, 2082560 pages, 789300 free.
2000 buffers, 1996 used, 0 dirty 0 wired down, repl age 573 0 w. io 0 w/crsr.
Disk Usage: 8764 reads avg 0 msec, 0% r 0% w last 0 s, 3073 writes,
29 read ahead, batch = 26. Autocompact 0 in 0 out, 0% saved.

So the number of buffers is defaulted to 2000 instead of the 230k I specified. Any idea what the reason for this could be? (ini file is in /var/lib/virtuoso/db)

Comment on Setting up a local DBpedia mirror with Virtuoso by joern

joern — Wed, 01 Feb 2012 09:34:05 +0000

Hi Nico,
with those settings virtuoso should use at least half of your ram and the rest should probably be OS buffers caching access to the hd… I’d just fire up top or htop and have a look on the virtuoso memory column. If it doesn’t use a significant amount of memory something is wrong. Are you sure you (re)started the virtuoso server after changing the buffer values? Did you set the MaxCheckpointRemap according to the post?
Cheers,
Jörn

Comment on Setting up a local DBpedia mirror with Virtuoso by Peter

Peter — Wed, 01 Feb 2012 09:23:41 +0000

Hi Joern,

thanks for the great tutorial! It saved my live! I don’t I’d have managed to get the server and the imports running without your valuable instructions.
Just one quick question, maybe someone can help with regard to this: I’m running the import for 2 (!) days now, an roughly have imported half of the files. Although I’m running on a pretty slow machine (dual core, 3 Gig Memory) I’m wondering that neither CPU nor memory are significantly in use. It’s almost exclusively the HDD which is working all the time, causing the import to be so incredibly slow probably. Any ideas where to tune things in the configuration?
My Virtuoso.ini config is curently set to but doesnt seem to have any effect though:
NumberOfBuffers = 224000
MaxDirtyBuffers = 200000

Thanks a lot!
Nico

Comment on Setting up a local DBpedia mirror with Virtuoso by mataron

mataron — Fri, 27 Jan 2012 14:16:46 +0000

Hi!
Thanks for the nice & helpful tutorial! It did actually save me from lots of searching.

Do you have any suggestion how to go about importing dbpedia 3.7? Is there any option in virtuoso to set the parser?

The error I get is similar to the one posted above.

Thanks in advance!

Comment on Setting up a local DBpedia mirror with Virtuoso by florian

florian — Tue, 17 Jan 2012 16:35:26 +0000

I really enjoyed following the tutorial!!!
I’m not a linux-expert but I found it easy to set up and it ran through without any errors. Because I don’t have a fast enought server I set up a virtual machine with VirtuelBox. The Virtuell Machine was running on 800MB RAM and a pretty old DualCore. It took 3 days to import the german version of dbpedia but Queries on the SPARQL-Endpoint work. You might experience Errors when querying the Endpoint because the estimated execution time exceeds a predefined limit. I fixed this by logging into http://your-server:8890 and increasing the limits in the SPARQL section of the admin-interface. Now its time to play

Greetings,
florian

Comment on Setting up a local DBpedia mirror with Virtuoso by joern

joern — Tue, 10 Jan 2012 14:11:38 +0000

uhm, i never noticed this, as i always imported the dump in one shot.

I noticed that things get terribly slow when you run out of RAM, so maybe that happens when you import the last part?
Also I’d do commits and checkpoints between the parts, just to make sure there’s not too much pending work.

Comment on Setting up a local DBpedia mirror with Virtuoso by June

June — Thu, 05 Jan 2012 03:00:28 +0000

Will rdf_loader_run() be more and more slower?
I load dbpedia by 4 parts, and when loading the last part, it is getting extraordinarily slow…

Comment on Duolingo: Learn a language and translate the web by joern

joern — Thu, 29 Dec 2011 16:01:31 +0000

Well, I always wanted to learn German or Spanish… but actually I’d also want to start with French.
Got a beta account?

Comment on Duolingo: Learn a language and translate the web by Raphael

Raphael — Thu, 29 Dec 2011 15:58:54 +0000

This is awesome. Can’t wait for support of languages I actually want to learn.