-
Notifications
You must be signed in to change notification settings - Fork 0
jtestard/COMP400
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
----------------------------- ZEN LIBRARY PROFILING ----------------------------- This project contains code for an extensive profiling of Derek Ruths's Zen library under memory constraints. My own profiling was done under Ubuntu 12.0.4 on an Asus computer with 8G RAM, 500GB of disk space and 4 2.2Ghz intel core i7 processors. However, these scripts should let you profile Zen under your own computer, provided it is using the Linux Operating system and you have super user access (only for the installation/configuration of the Cgroups). The profiling methods used are specific to the Linux operating system. It is not possible to use the C and python files included here on another ooperating system. Profiling data is generated using the following method. 1) Setup the Cgroups environment: Cgroups is a Linux technogy (orignally unix, so maybe MAC OS X can use them as well) that allows user to specify how much maximum virtual or physical memory they want a process to use. This library may not be installed on your computer by default. The next script assumes that the Cgroups tool has been installed. If not, please use the following command to install it first: sudo apt-get install cgroups Cgroups configuration can be done using the following command from the project root directory (note that superuser access may be required): sh sh/cgconfig.sh 2) Generating the graphs : This is the only step that is not operating-system specific. You can generate graphs for the 5 graphs categories profiled by using the following command. sh sh/generate_graphs This operation takes time, because the graphs generated can be quite large (up to 300 MB), and a total of 100 graphs are generated. The operation should take about 40 minutes (most of the time is spent on the metric graphs category, this is the only graph category not to be written in Cython :s). Information about generating times and IO saving times as well as graph size are available in csv format under the /csv folder. 3) Profiling C : This step shows how the computer reacts to C memory allocations under the Cgroups constraints. Graphs describing the virtual memory and RAM usage when using C are available under the /csv folder. To obtain the C profiling, please execute the following command from the project's root directory : sh sh/c_start.sh 4) Profiling Zen : This step profiles the load time and query access time of the Zen library on each type of graph. Profiling results are saved in .csv files available under /csv. To obtain the Zen library profiling under the CGroups constraints, please execute the following command fro mthe project's root directory : sh sh/python_start.sh
About
Representation of graph databases optimized for k-th neighbour search on large graphs stored on hard disk
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published