ROCKS 5.3 Notes



List installed rolls

rocks list roll

Install a roll (on a running system)

You’ll usually get an ISO file. Let’s call it rocks-roll.iso

# Appending clean=1 will remove any older rolls of same name  
rocks add roll rocks-roll.iso  
# This should now show the newly added roll  
rocks list roll  
# Enable the roll  
rocks enable roll sge  
# Rebuild the distro  
cd /export/rocks/install  
rocks create distro  
# Run the roll; generates a setup shell script  
rocks run roll rocks-roll > /tmp/  
sh -x /tmp/

Note: When installing the SGE roll, I had errors about not finding the “TRANS.TBL” file. Manually copying it /export/rocks/install/rolls/sge/5.3/x86_64/RedHat/RPMS solved the problem. Strange…

Remove a roll

# Disable and remove the roll  
rocks disable roll sge  
rocks remove roll sge  
# Rebuild the distro  
cd /export/rocks/install  
rocks create distro


Running a command on all nodes

This is far better than some bash cleverness since it’s done simultaneously (and not sequentially).

rocks run host command="yum -y install gcc44*"

Testing the Kickstart File

/state/partition1/rocks/install/sbin/kickstart.cgi -c compute-0-1 >  
# This is better  
rocks list host profile compute-0-1 > compute-0-1.xml

Installing a Node

# Wait for the node to come up, for ROCKS to assign it a name & IP,  
# and for an asterisk like "(*)"

Assign a desired name to a node

insert-ethers does things very sequentially. If you have another set of nodes that you want to number differently:

insert-ethers --rack=1 --rank=14

This will give your appliance a name like compute-1-14

Reinstalling a Node

Assume that compute-0-1 is the node under consideration.

If node is available

On the node to be reinstalled, run


You can also shoot it from the frontend

shoot-node compute-0-1

If node is unavailable

rocks set host boot compute-0-1 action=install  
# After the node has kickstarted; not doing this will result in infinite install loop  
rocks set host boot compute-0-1 action=os

Removing a Node

rocks remove host compute-0-9

Installing RPMs

I’m using ROCKS 5.3 on an x86_64 server.

Change IP address

rocks set host interface ip compute-1-0 iface='eth0' ip=''

See all IP and MAC addresses

rocks list host interface

Track usage

qacct -h

Removing the “-h” flag will give you the total system usage. This command has a lot of switches and is quite flexible (e.g. per user, per node, per job, since 8 days)


Starting the SSH Agent

You may get this error when trying to shoot a node:

[root@cluster ~]# shoot-node compute-2-1  
Shoot Node - version 5.5  
Usage:  /opt/rocks/sbin/shoot-node [-h] host ...  
Requires ssh-agent to launch

To start the SSH agent,

ssh-agent $SHELL  


ROCKS runs an alternate MySQL server on port 40000. It’s used to maintain the SGE and Ganglia (amongst other) databases. Here’s the full process:


Luckily, you can turn it on and off like a SysV service:

service foundation-mysql stop  
service foundation-mysql start

You can also see what’s in the cluster database by connecting to the socket specified:

mysql --socket=/var/opt/rocks/mysql/mysql.sock --user=rocksdb

Kickstart Params

View list of params with

rocks list attr

Here’s how you change the Kickstart_PublicAddress attribute

rocks set attr Kickstart_PublicAddress


I once needed to change the password to the Wordpress administration page. The ROCKS manual said I should’ve used admin and the cluster’s root password. This didn’t work, presumably since I changed the root password later.

I was able to reset the Wordpress admin password by connecting using this:

mysql -uwordpress -p -hlocalhost --socket=/var/opt/rocks/mysql/mysql.sock

and issuing this:

UPDATE wordpress.wp_users
SET user_pass = md5('PASSWORD')
WHERE user_login = 'admin';