Curator

Benchmark new hp cloud compute instance

2012-04-26T00:00:00+00:00

Today I got around to beta testing the new hp cloud. The hp cloud is powered by OpenStack. The management console allows you to quickly launch an instance; if you’re familiar with the AWS Console you’ll feel right at home.

Here’s a UnixBench Benchmark on their standard.small 2vCPU/2GB RAM/60GB HD - CentOS 64bit Image



========================================================================
   BYTE UNIX Benchmarks (Version 5.1.3)

   System: server---02012-04-26-162925-0000.localdomain: GNU/Linux
   OS: GNU/Linux -- 2.6.32-220.2.1.el6.x86_64 -- #1 SMP Fri Dec 23 02:21:33 CST 2011
   Machine: x86_64 (x86_64)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: QEMU Virtual CPU version 0.14.0 (5333.5 bogomips)
          x86-64, MMX, Physical Address Ext, SYSCALL/SYSRET
   CPU 1: QEMU Virtual CPU version 0.14.0 (5333.5 bogomips)
          x86-64, MMX, Physical Address Ext, SYSCALL/SYSRET
   14:38:45 up  5:07,  1 user,  load average: 0.00, 0.00, 0.00; runlevel 3

------------------------------------------------------------------------
Benchmark Run: Thu Apr 26 2012 14:38:45 - 15:07:02
2 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       19422142.7 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     2822.4 MWIPS (9.9 s, 7 samples)
Execl Throughput                               1720.6 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        298470.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          121086.2 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        961321.9 KBps  (30.0 s, 2 samples)
Pipe Throughput                              651916.4 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 127890.3 lps   (10.0 s, 7 samples)
Process Creation                               4264.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   3240.7 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    657.8 lpm   (60.1 s, 2 samples)
System Call Overhead                        1154673.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   19422142.7   1664.3
Double-Precision Whetstone                       55.0       2822.4    513.2
Execl Throughput                                 43.0       1720.6    400.1
File Copy 1024 bufsize 2000 maxblocks          3960.0     298470.7    753.7
File Copy 256 bufsize 500 maxblocks            1655.0     121086.2    731.6
File Copy 4096 bufsize 8000 maxblocks          5800.0     961321.9   1657.5
Pipe Throughput                               12440.0     651916.4    524.0
Pipe-based Context Switching                   4000.0     127890.3    319.7
Process Creation                                126.0       4264.7    338.5
Shell Scripts (1 concurrent)                     42.4       3240.7    764.3
Shell Scripts (8 concurrent)                      6.0        657.8   1096.4
System Call Overhead                          15000.0    1154673.2    769.8
                                                                   ========
System Benchmarks Index Score                                         688.9

------------------------------------------------------------------------
Benchmark Run: Thu Apr 26 2012 15:07:02 - 15:35:22
2 CPUs in system; running 2 parallel copies of tests

Dhrystone 2 using register variables       34878089.5 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     5516.2 MWIPS (10.0 s, 7 samples)
Execl Throughput                               3917.0 lps   (29.8 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        482047.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          132102.6 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1278367.3 KBps  (30.0 s, 2 samples)
Pipe Throughput                             1542930.2 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 268617.7 lps   (10.0 s, 7 samples)
Process Creation                              12060.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   5236.8 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    711.7 lpm   (60.1 s, 2 samples)
System Call Overhead                        1949027.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   34878089.5   2988.7
Double-Precision Whetstone                       55.0       5516.2   1002.9
Execl Throughput                                 43.0       3917.0    910.9
File Copy 1024 bufsize 2000 maxblocks          3960.0     482047.7   1217.3
File Copy 256 bufsize 500 maxblocks            1655.0     132102.6    798.2
File Copy 4096 bufsize 8000 maxblocks          5800.0    1278367.3   2204.1
Pipe Throughput                               12440.0    1542930.2   1240.3
Pipe-based Context Switching                   4000.0     268617.7    671.5
Process Creation                                126.0      12060.7    957.2
Shell Scripts (1 concurrent)                     42.4       5236.8   1235.1
Shell Scripts (8 concurrent)                      6.0        711.7   1186.1
System Call Overhead                          15000.0    1949027.6   1299.4
                                                                   ========
System Benchmarks Index Score                                        1199.5

Benchmark details:


wget http://byte-unixbench.googlecode.com/files/UnixBench5.1.3.tgz
tar -zxvf UnixBench5.1.3.tgz
cd UnixBench
make
./Run

Downloading 1gb oracle packages directly to your server.

2012-02-03T00:00:00+00:00

Using the cookie from your logged in session you can use wget to download the packages on your server.

Get the Google Chrome plugin
login to Oracle
hit the cookie.txt export icon (upper right)
create a file cookies.txt on your server and copy the contents of the new window into file.
wget -x –load-cookies cookies.txt http://download.oracle.com/otn/bea/jrockit/jrockit-jdk1.6.0_29-R28.2.2-4.1.0-linux-ia32.bin

make sure to accept the license agreement on the oracle site to enable the correct download links.

Benchmark pigz vs gzip

2011-03-24T00:00:00+00:00

What is pigz ?

“pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data.” – pigz

Who doesn’t love benchmarks ?

File Data: Tarred Enron Corpus
File Size: 1823139840 bytes ~ 1.7 gigabytes

System Specs:
Processor Name: Intel(R) Xeon(R) CPU E5507
Processor Speed: 2.7 GHz
Number Of Processors: 1
Total Number Of Cores: 4
L2 Cache (per core): 256 KB
L3 Cache: 4 MB

pigz results

compress

time pigz -9 enron_mail_082109.tar
real 0m44.136s
user 2m5.400s
sys 0m1.570s

decompress

time pigz -d -9 enron_mail_082109.tar.gz
real 0m14.354s
user 0m12.270s
sys 0m3.260s

gzip results

compress

time gzip -9 enron_mail_082109.tar
real 2m27.748s
user 1m53.860s
sys 0m1.380s

decompress

time gunzip -9 enron_mail_082109.tar.gz
real 0m29.564s
user 0m18.270s
sys 0m2.250s

Here’s another benchmark

Linux Tools: fuser

2010-11-06T00:00:00+00:00

Find what process is listening on a specific port

List all listening ports

netstat -l
Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 :9090:* LISTEN
tcp 0 0 :nrpe:* LISTEN

fuser is given the (-n) tcp namespace

$sudo fuser -n tcp 9090

The output is a list of processes associated with that port , 9090 in this example.

9090/tcp: 8794 26222 26223 26224 26225 26226 26227 26228 26229 26230 26231

Who’s accessing that mount ?

Sometimes when you’re trying to unmount a drive you’ll get a message saying the device is busy. With fuser you can find the process accessing the device.

fuser -v (verbose) -m (mounting point) /path/to/mount

$sudo fuser -v -m /mnt/nfsshare/
…………………USER PID ACCESS COMMAND
/mnt/nfsshare/: edwin 25740 ..c.. bash

It was me!

Take a look at the fuser man pages for information on the different types of access.

Using Jumbo Frames w/ MySQL

2010-10-16T00:00:00+00:00

I first learned about jumbo frames while working on a MySQL cluster deployment. The MySQL cluster required low latency to keep the nodes in sync. The budget didn’t allow for the recommended infiniband hardware. We decided to swap out an older Dell switch (used for the internal backend network 10.0.0.0/8 ) with a Cisco switch that supported 9000 byte frames (Jumbo Frames).

All the MySQL nodes were built with dual GigE network cards and we wanted to take advantage of that. Ethernet uses a standard frame size of 1500 bytes. We configured our network interfaces to use jumbo frames by setting the MTU with the following command.

ip link set eth0 mtu 9000

This allows larger frames to be transferred between nodes improving the local network throughput. Once the jumbo frames were configured, we were able to stand up an operational 4 node MySQL Cluster. We learned a few things through trial and error. The database size increased significantly when moving to MySQL Cluster. A varchar(255) became a char(255); a fixed-length character type.

Conclusion

Jumbo frames work well for transferring large files; make sure your network and destination support it.

The implementation of the MySQL cluster allowed us to work through some limitations of the version, 5.0 at the time, but from an operational stand point we didn’t feel confident in the solution. The entire database needed to be stored in RAM and recovering from a failed node was very slow. We decided on a master/slave setup with a 2nd slave for running reports.

Copying a table in MySQL

2010-09-03T00:00:00+00:00

CREATE TABLE new_table LIKE production.table;
INSERT new_table SELECT * FROM production.table;

If you need a MySQL backup solution, take a look at zmanda

Source

Ec2 On-Demand vs Reserved Pricing

2010-05-08T00:00:00+00:00

Cloud, VPS, Grid - take your pick. AWS continues to offer new useful services (EBS, CloudFront, BootEBS). Yes, I’m a big fan.

I’ve crunched the numbers and have one year worth of pricing data (as of May 06 2010) for the small, large, and xlarge instance. Hopefully someone using EC2 services will find this helpful in planning their next steps. I didn’t include EC2 Spot instances because pricing on that type is not guaranteed.

The data used to generate these graphs can be downloaded here.

small OD large OD xlarge OD
small RS large RS xlarge RS

Interesting points

These are priced about the same.
240 days of small OD = 365 days of small R
240 days of large OD = 365 days of large R
240 days of xlarge OD = 365 days of xlarge R

On the 172 day of use, On-Demand pricing is about the same as Reserved Pricing.

Improving your resolv.conf file

2010-04-30T00:00:00+00:00

The resolver can load up to 3 name servers. If your server needs to find the mx record of gmail.com, it will first need to resolve gmail.com locally.

using a resolver from resolv.conf
dig @8.8.8.8 gmail.com -t mx +short

result: alt1.gmail-smtp-in.l.google.com
connect to alt1.gmail-smtp-in.l.google.com
dig @8.8.4.4 alt1.gmail-smtp-in.l.google.com +short

result: 209.85.227.27
connect to 209.85.227.27:25

Optimized resolv.conf

nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 4.2.2.2
option rotate
option timeout:1

option rotate - load balance the queries across the 3 listed servers
option timeout:1 - sets the lookup timeout to 1 sec (default 5s)