[Tig] Can someone help me with my drive array?

[Michael Russell]

Bob,
   I'll admit that I know nothing about Sun's StorageTek box.  I'll also admit
I have never used iozone.  You are now thinking - what the hell is the point
of this guy replying?  Well, hopefully I can give you a few thoughts anyway
since we have tested many drives and configurations over the years.
   To maximize performance:
     1) You want to have write-caching enabled on the drives.  Naturally, this
        has the potential for data loss depending on the battery backup of
        the RAID controller, drives and probably some other factors.
        However, based on your results (ballpark 180MB/s write & 330 MB/s
        read), I'd say this is one of your issues.  You should discuss this
        issue with Sun to see what the potential for data loss is and whether
        it's OK for your situation.
     2) There may be other RAID controller settings related to write caching
        optimization.
     3) You want to make sure that the channel is not saturated when you
        have other options.  For example, I'd guess that each of the drives
        can do about 80 MB/s, so 12 drives could theoretically (very, very
        theoretically) do about 960 MB/s in a RAID 0 configuration.  This, is
        of course more than a single 4Gb FC channel can handle.  However, if
        you split the drives up so that 6 are on each of your 2 channels,
        you'll avoid this problem.  Depending on the RAID controller / switch /
        box etc., this may be anywhere from simple to impossible to do.
   I hope some of this helps.

-- 
                                         Michael Russell
                                         russell at mathtech.com
                                         MTI Film, LLC
                                         Providence, RI  02906  USA

Bob Friesenhahn wrote:
> TIGNABFS08 - NAB for 2008
> http://colorist.org/wiki3/index.php/TIGNABFS08
> Thanks to The Chimney Pot for support in 2008
>  ====
> 
> 
> I am setting up a drive array (Sun StorageTek 2540) which is fitted 
> with 12 300GB 15KRPM SAS drives.  There are dual controllers and dual 
> 4Gbit FC connections to my host.  This has been a long time comming 
> and at times I experienced the agony of defeat (first unit was smashed 
> in shipping), but it is finally here and apparently working.  The 
> drive array is in my home, which is probably an unusual location.  The 
> drive array is a present to myself to support GraphicsMagick 
> development.
> 
> I have spent several days now working to make sure that the fiber 
> channel connection is solid (it was bouncing) and to understand 
> performance issues.  Unfortunately, I don't understand the performance 
> values that I am seeing.  It seems that regardless of what RAID 
> configuration I use (even RAID 0 with 12 drives), the single-file 
> write I/O rate (according to 'iozone') is in the range of 170-200 
> MB/second.  I was hoping to do 1/3 better than that.  There is a 
> bottleneck somewhere.  An odd thing, is that other substantial 
> I/O activity on the filesystem does not seem to impact iozone results.
> 
> I don't know exactly what iozone is doing, but every once in a while 
> the disk activity (as reported by the OS) seems to cease for 10 or 20 
> seconds, yet the numbers printed by iozone are fairly consistent.
> 
> Can someone here with experience with SAN and drive array performance 
> help me understand what is going on and the performance levels I 
> should actually expect?  The output from an iozone run is attached.
> 
> Thanks,
> 
> Bob
> ======================================
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
> 
>  	Iozone: Performance Test of File I/O
>  	        Version $Revision: 3.283 $
>  		Compiled for 64 bit mode.
>  		Build: Solaris10gcc-64
> 
>  	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
>  	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
>  	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
>  	             Randy Dunlap, Mark Montague, Dan Million,
>  	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
>  	             Erik Habbinga, Kris Strecker, Walter Wong.
> 
>  	Run began: Thu Feb 14 08:52:33 2008
> 
>  	Auto Mode
>  	Using Minimum Record Size 64 KB
>  	Using Maximum Record Size 512 KB
>  	Using minimum file size of 16777216 kilobytes.
>  	Using maximum file size of 33554432 kilobytes.
>  	Command line used: iozone -a -i 0 -i 1 -y 64 -q 512 -n 16G -g 32G
>  	Output is in Kbytes/sec
>  	Time Resolution = 0.000001 seconds.
>  	Processor cache size set to 1024 Kbytes.
>  	Processor cache line size set to 32 bytes.
>  	File stride size set to 17 * record size.
>                                                              random  random    bkwd  record  stride
>                KB  reclen   write rewrite    read    reread    read   write    read rewrite    read   fwrite frewrite   fread  freread
>          16777216      64  196815  115715   495561   382353
>          16777216     128  180500  179898   333415   332368
>          16777216     256  179846  179635   333178   333090
>          16777216     512  177443  181405   332257   346124
>          33554432      64  192275  112841   332105   332730
>          33554432     128  179471  174863   331858   332041
>          33554432     256  177570  194285   332262   332414
>          33554432     512  200085  179983   331898   331971
> 
> iozone test complete.
> 
> _______________________________________________
> No personal abuse; absolutely no advertising or marketing on the main TIG mailinglist.  Emergency
> tig contact address is rob (at) calarts.edu
> Tig mailing list
> Tig at tig.colorist.org
> http://tig.colorist.org/mailman/listinfo/tig