CentOS 6 – Voldemort install

EnglishLinuxTutorials

Voldemort is a NoSQL database, a distributed key-value storage system… a kind of hashmap (for java guys…).

URL resources:
– http://project-voldemort.com/
– https://github.com/voldemort/voldemort/downloads
– https://github.com/voldemort/voldemort/wiki

Your base server can be for instance a CentOS 6 Net install installation in “Basic server” mode.

Let’s update everything:

1	`yum update -y`

We need a Java Virtual Machine:

1	`yum` `install` `-y java-1.6.0-openjdk`

And then let’s download the Voldemort project:

cd /usr/local/
wget -c https://github.com/downloads/voldemort/voldemort/voldemort-0.90.tar.gz
tar zxvf voldemort-0.90.tar.gz
rm -f voldemort-0.90.tar.gz
ln -s voldemort-0.90 voldemort

Just prepare some configuration file:

cp /usr/local/voldemort/config/single_node_cluster/config/* /usr/local/voldemort/config/

And we affects rights:

groupadd voldemort
useradd -g voldemort voldemort
chown -R voldemort:voldemort /usr/local/voldemort
chown -R voldemort:voldemort /usr/local/voldemort-0.90

We need a little startup script:

cat > /etc/init.d/voldemort << "EOF"
#!/bin/bash
# chkconfig: 3 21 91
#
# Voldemort
#
# description: Start up the Voldemort NoSQL DB.
 
# Source function library.
. /etc/init.d/functions
 
RETVAL=$?
VOLDEMORT_HOME=/usr/local/voldemort
VOLDEMORT_USER=voldemort
 
case "$1" in
  start)
    if &#91; -f $VOLDEMORT_HOME/bin/voldemort-server.sh &#93;;
      then
      echo $"Starting Voldemort"
      /bin/su - $VOLDEMORT_USER -c $VOLDEMORT_HOME/bin/voldemort-server.sh > $VOLDEMORT_HOME/voldemort.log 2>&1 &
    fi
  ;;
 
  stop)
    if [ -f $VOLDEMORT_HOME/bin/voldemort-stop.sh ];
      then
      echo $"Stopping Voldemort"
      /bin/su - $VOLDEMORT_USER -c $VOLDEMORT_HOME/bin/voldemort-stop.sh
    fi
  ;;
 
  *)
    echo $"Usage: $0 {start|stop}"
    exit 1
  ;;
esac
 
exit $RETVAL
EOF

Let’s make it executable:

1	`chmod` `a+x` `/etc/init.d/voldemort`

And we add a VOLDEMORT_HOME in the /etc/bashrc file:

cat >> /etc/bashrc << EOF export VOLDEMORT_HOME=/usr/local/voldemort EOF [/shell] We export this line in our current terminal in order to use it now: [shell]export VOLDEMORT_HOME=/usr/local/voldemort[/shell] And we launch voldemort: [shell]/etc/init.d/voldemort start[/shell] If you want your server to execute it at boot time (and kill it before shutdown): [shell] /sbin/chkconfig --level 3 voldemort on /sbin/chkconfig --level 06 voldemort off [/shell] If you need to add rules to the firewall: [shell] iptables -A INPUT -p tcp --dport 6666 -j ACCEPT iptables -A INPUT -p tcp --dport 6667 -j ACCEPT iptables -A INPUT -p tcp --dport 8081 -j ACCEPT /sbin/service iptables save [/shell] And that's it.. your Voldemort server is up and running (quite easy actually). Configuration

Everything is nicely described here: http://project-voldemort.com/configuration.php (beware, some XML tags are not closed correctly in their samples).

For my tests I’m using virtualized servers inside VirtuaBox, network cards are cofigured with “Host only networking”, so it’s easy to make them talk together… (my IP adresses are 192.168.56.101 and 192.168.56.102)

Here is the /usr/local/voldemort/config/cluster.xml file I use:

<cluster>
    <name>VBOX_JU_CLUSTER</name>
    <zone>
      <zone-id>0</zone-id>
      <proximity-list>1</proximity-list>
    </zone>
    <zone>
      <zone-id>1</zone-id>
      <proximity-list>0</proximity-list>
    </zone>
    <server>
      <id>0</id>
      <host>192.168.56.101</host>
      <http-port>8081</http-port>
      <socket-port>6666</socket-port>
      <admin-port>6667</admin-port>
      <partitions>0,1,2,3</partitions>
      <zone-id>0</zone-id>
    </server>
    <server>
      <id>1</id>
      <host>192.168.56.102</host>
      <http-port>8081</http-port>
      <socket-port>6666</socket-port>
      <admin-port>6667</admin-port>
      <partitions>4,5,6,7</partitions>
      <zone-id>1</zone-id>
    </server>
</cluster>

You copy this file on your 2 servers, they must be identical on every server. Then we have to configure our specific /usr/local/voldemort/config/server.properties for each server, you must adapt node.id and max.thread. node.id is sequential, unique, and starts with 0:

– Serveur 01

node.id=0
max.threads=42
...

node.id=0

max.threads=42

...

– Serveur 02

node.id=1
max.threads=142
...

node.id=1

max.threads=142

...

For now we will work only with one data store, and you configure it inside /usr/local/voldemort/config/stores.xml file:

test
2

2</preferred-reads>
        <required-reads>1</required-reads>
        <preferred-writes>2</preferred-writes>
        <required-writes>1</required-writes>
        <persistence>bdb</persistence>
        <routing>client</routing>
        <routing-strategy>consistent-routing</routing-strategy>
        <key-serializer>
            <type>string</type>
        </key-serializer>
        <value-serializer>
            <type>string</type>
        </value-serializer>
    </store>
</stores>
[/xml]

Servers doesn't need to have the same <code>/usr/local/voldemort/config/stores.xml</code> file (I put the same in my example), because you can configure each store depending on zone replication factors... more detail here: <a href="http://project-voldemort.com/configuration.php" title="http://project-voldemort.com/configuration.php" target="_blank">http://project-voldemort.com/configuration.php</a>

<strong>Test our servers</strong>

We now need to test that everything works fine.
From one of your server (or from anywhere that sees your servers):
[shell]/usr/local/voldemort/bin/voldemort-shell.sh test tcp://192.168.56.102:6666[/shell]

You are now connected to the <strong>test</strong> store, and you can insert data.:
<pre>
> get "key02"
null
> put "key01" "value 01"
> put "key02" "value 02"
> get "key01"
version(0:1): "value 01"

2</preferred-reads>

<required-reads>1</required-reads>

<preferred-writes>2</preferred-writes>

<required-writes>1</required-writes>

<routing>client</routing>

<routing-strategy>consistent-routing</routing-strategy>

<key-serializer>

<type>string</type>

</key-serializer>

<value-serializer>

<type>string</type>

</value-serializer>

</store>

</stores>

[/xml]

Servers doesn't need to have the same <code>/usr/local/voldemort/config/stores.xml</code> file (I put the same in my example), because you can configure each store depending on zone replication factors... more detail here: <a href="http://project-voldemort.com/configuration.php" title="http://project-voldemort.com/configuration.php" target="_blank">http://project-voldemort.com/configuration.php</a>

<strong>Test our servers</strong>

We now need to test that everything works fine.

From one of your server (or from anywhere that sees your servers):

[shell]/usr/local/voldemort/bin/voldemort-shell.sh test tcp://192.168.56.102:6666[/shell]

You are now connected to the <strong>test</strong> store, and you can insert data.:

<pre>

> get "key02"

null

> put "key01" "value 01"

> put "key02" "value 02"

> get "key01"

version(0:1): "value 01"

Let’s check where the data are currently saved :

> preflist "key01"
Node 1
host:  192.168.56.102
port: 6666
available: yes
last checked: 1314187968521 ms ago

Node 0
host:  192.168.56.101
port: 6666
available: yes
last checked: 1314187968521 ms ago

> preflist "key01"

Node 1

host: 192.168.56.102

port: 6666

available: yes

last checked: 1314187968521 ms ago

Node 0

host: 192.168.56.101

port: 6666

available: yes

last checked: 1314187968521 ms ago

This means your data is saved on both server ! Why ? Just look inside your /usr/local/voldemort/config/stores.xml file, we have :
– replication-factor = 2
– preferred-reads = 2
– required-reads = 1
– preferred-writes = 2
– required-writes = 1
Pretty clear no !

See you…

Quidquid

Development stuff