V890 running Solaris 10

Log entries | KWIC index | Other Logs

Sat, 18 Mar 2006 17:57



Top   Next   Prev   End

It's time to start thinking about upgrading our current server. E450 end-of-life is in 2007, so we need a new mid-range box with lots of storage.

    http://www.sun.com/servers/index.jsp?cat=Sun%20Fire%20Midrange%20Servers&tab=3
    Midrange Servers

    Designed for compute density backed by enterprise-class features,
    Sun's midrange servers deliver high performance and protect
    your IT investments over time.  Sun has enhanced the industry's
    leading 64-bit server line with the UltraSPARC IV+ processor,
    offering up to five times the performance of previous UltraSPARC
    III systems.  Yet you retain Sun's migration-enabling unbroken
    binary application compatibility with the Solaris OS.

    Processors: 4 - 24 processors, 8 - 48 threads
    Operating Systems: Solaris

    - Sun Fire V490 Server
    - Sun Fire V890 Server
    - Sun Fire E2900 Server
    - Sun Fire E4900 Server
    - Sun Fire E6900 Server
    - Sun Fire V480 Server
    - Sun Fire V1280 Server

    Starting at $30,995
     
Sat, 18 Mar 2006 18:08



Top   Next   Prev   End
    Sun Fire V890 Server

                        SMALL               MEDIUM
    UltraSPARC IV+      2x1.5 GHz           4x1.5 GHz

    L2 On Chip Cache
    per Processor       2 MB                2 MB

    L3 External Cache
    per Processor       32 MB               32 MB

    Memory              8 GB (16x512-MB)    16 GB (32x512-MB)

    10000 RPM FC-AL
    Disk Drive          4x146 GB            4x146 GB

    DVD-ROM             1                   1
    Gigabit Ports       1                   1
    Ethernet Port       1x10/100BASE-T      1x10/100BASE-T
    Serial Port         1                   1
    USB Port            2                   2
    PCI Slot            9                   9
    Power Supply        3 (N+1)             3 (N+1)

    Remote Management
    Software            1                   1

    List Price          $34,995.00          $62,995.00

    Java Enterprise System Pre-Installed
    Solaris 10 Pre-Installed
    Warranty: 3 Years, Next Business Day
     

This can hold up to 12 drives.

Sat, 18 Mar 2006 18:10



Top   Next   Prev   End

The SunFire V490 was a little smaller, but the price for a medium setup was still over $50K. Anything else was either too small or too expensive. We want lots of disk drives; at least 12-20 slots, 146 GB each.

We currently have 16 18-GB drives, for a total of 288 GB.

Sat, 18 Mar 2006 18:19



Top   Next   Prev   End

V440 looks about right, with a mixed configuration. We could use 8 GB of memory, but I want the bigger drives. 4 146-GB drives gives us 584 GB of space just inside the main box.

    Sun Fire V440

                        MEDIUM         LARGE         EXTRA LARGE
    UltraSPARC IIIi
    CPU                 4x1.593 GHz    4x1.593 GHz   4x1.593 GHz

    Internal Cache/CPU  1 MB           1 MB          1 MB

    Memory              8 GB           16 GB         32 GB
                        (16x512-MB)    (16x1-GB)     (16x2-GB)

    10000 RPM Ultra320
    SCSI Disk Drive     4x73 GB        4x73 GB       4x146 GB

    DVD-ROM Drive       1              1             1

    10/100/1000 Mb/s
    Ethernet Port       2              2             2

    Serial Port         1 - DB9        1 - DB9       1 - DB9
    USB Port            4              4             4
    Power Supply        2 (1+1)        2 (1+1)       2 (1+1)
    List Price          $23,995.00     $30,995.00    $40,995.00

    All:
    Ships within 11 Business Days
    Solaris 10 Pre-Installed
    Java Enterprise System Pre-Installed
     

NOTE: The CPUs are not dual-core.

Sat, 18 Mar 2006 18:35



Top   Next   Prev   End

Would we be better off with a storage array?

    http://store.sun.com/CMTemplate/CEServlet?process=SunStore&cmdViewProduct_CP&catid=112508
    Sun StorEdge 3120 SCSI Array (LARGE)

    10000 RPM SCSI Disk Drive     1200 GB (4x300 GB)
    LVD Host Connection           2
    Power Supply                  2 AC plus 2 fans
    Placement                     Rack-Ready
    Ships Within                  6 Business Days
    List Price                    $10,495.00
     

We currently have 2 GB memory:

    me% prtconf
    System Configuration:  Sun Microsystems  sun4u
    Memory size: 2048 Megabytes
    System Peripherals (Software Nodes):
    ...
     

If we got a medium V440 plus two 3120 arrays, we'd spend around $45,000 for 4x the memory we have now, 4x faster CPUs, and a little over 2400 GB disk space (8x what we have now).

Thu, 23 Mar 2006 04:17



Top   Next   Prev   End

Problem with fans?

    Message-ID: <442267A5.1080206@ifrance.com>
    Date: Thu, 23 Mar 2006 10:17:25 +0100
    From: Rami Aubourg-Kaires <rami.aubourg@ifrance.com>
    To: sunmanagers@sunmanagers.org
    Subject: Sunfire V210 PSU failures

    We've had two fan failures within a week on the power supplies
    of two identical Sunfire V210's.  Nothing dramatic, since the
    PSU works fine for quite a while without the fan, given the
    amount of fans (and noise!)  you have in this server.

    However, Googling for the same problem popped up quite a lot
    of similar problems with Sunfire V210's.

    I was wondering whether other users had had the the same type
    of problems with V210's or V240's, and whether it could be
    recurrent even after the power supply replacement.
     

Maybe not.

    Date: Fri, 24 Mar 2006 09:51:04 +0100
    From: Rami Aubourg-Kaires <rami.aubourg@ifrance.com>
    To: sunmanagers@sunmanagers.org
    Subject: SUMMARY: Sunfire V210 PSU failures

    Hello, and thanks to all who replied.

    I received eight replies to this, and seven of them reported
    the same problem on several V210's (no V240's).  One person
    who had 5 V210's has had no problems whatsoever.  One person
    said there was a FIN coming out on this problem.

    No one has had any problems with the replacement units.
    It seems that there was a bad batch on these PSU's.
     
Tue, 2 May 2006 16:40



Top   Next   Prev   End

Our best bet looks like a V890 server with two UltraSparc IV+ CPUs; they're dual-core, so this is like having four single-core CPUs.

A V440 plus a storage array will cost more than a V890 plus internal drives. Also, a V440 or V490 plus an array has a large footprint; we need at least a 6-foot rack.

A V890 can handle up to 12 146-Gb drives (1.7 terabytes). These drives are fiber-channel with a new architecture; we don't need multiple I/O controllers like we have with our E450. The system puts out 11,000 BTU max.

At some point in the future, 300-Gb drives may be certified for use with the system. This is demand-driven, so it'll probably be a year or two down the line.

The operating system would be Solaris 10; we're using Solaris 8. 10's been stable and in production use for about a year.

We start with 8-Gb of memory, and we can go up by 2-Gb increments if necessary.

Our current network (100 Mbit) is supported.

The fans on V890s are completely different, so we won't have the problems mentioned earlier.

Mon, 8 May 2006 16:05



Top   Next   Prev   End

Started looking over some of the Solaris-10 documentation, and found some nice improvements:

    * IPF (packet filter) is now included.
    * Support for in-kernel caching of web pages.
    * RBAC to reduce root privileges.
    * ASET and Solaris Security Toolkit for security enhancement.
    * Monitoring of machine resource use and file integrity.
     

When the system is first installed, use BART to create a manifest of every file. This is similar to "mtree".

System performance monitoring is pretty much the same; use iostat, etc.

FIXME: start running "sar" from the moment the system is installed; p395 in "Advanced system admin" manual.

Mon, 8 May 2006 19:48



Top   Next   Prev   End

FIXME: remove VM tuning stuff from the production /etc/system.

The configuration mechanism of the paging subsystem was changed, starting in the Solaris 9 release. Instead of depending on a set of predefined values for fastscan, slowscan, and handspreadpages, the system determines the appropriate settings for these parameters at boot time. Setting any of these parameters in the /etc/system file can cause the system to use less than optimal values.

Ignore cachefree as well; it's set at boottime by the system.

Mon, 8 May 2006 19:55



Top   Next   Prev   End

FIXME: set maxphys to 1048576?

You might also consider changing this parameter when doing I/O to and from a UFS file system where large amounts of data (greater than 64 Kbytes) are being read or written at any one time. The file system should be optimized to increase contiguity. For example, increase the size of the cylinder groups and decrease the number of inodes per cylinder group. UFS imposes an internal limit of 1 Mbyte on the maximum I/O size it transfers.

Mon, 8 May 2006 20:10



Top   Next   Prev   End

FIXME: set segmap_percent higher, say 20% ?

If heavy file system activity is expected, and sufficient free memory is available, you should increase the value of this parameter.

Mon, 8 May 2006 20:13



Top   Next   Prev   End

Don't set ufs_LW and ufs_HW; they're pretty big as it is.

The value of the interrupt blanking parameter is a trade-off between network throughput and processor utilization. If higher processor utilization is acceptable for achieving higher network throughput, then disable interrupt blanking. If lower processor utilization is preferred and higher network latency is the penalty, then enable interrupt blanking.

Mon, 8 May 2006 20:45



Top   Next   Prev   End

In the Solaris software, all System V IPC facilities are either automatically configured, or the facilities can be controlled by resource controls. These facilities might be shared memory, message queues, or semaphores. Resource controls allow IPC settings to be made on a per-project or per-user basis on the local system or in a name service environment.

In previous Solaris releases, IPC facilities were controlled by kernel tunables. You had to modify the /etc/system file and reboot the system to change the default values for these facilities. Because the IPC facilities are now controlled by resource controls, their configuration can be modified while the system is running. Many applications that previously required system tuning to function might now run without tuning because of increased defaults and automatic allocation of resources.

Mon, 8 May 2006 20:49



Top   Next   Prev   End

We can sign compiled binaries!

The libraries and executable files in the Solaris 10 OS include digital signatures that can be used to verify the integrity of these files. The digital signature provides a way to detect any accidental change or deliberate tampering with the executable content of the file.

Plug-ins for the Solaris Cryptographic Framework are automatically verified when loaded by the system. The elfsign command can be used manually to verify any signed file. Developers and administrators can also use elfsign to sign their own code. For further information, see the elfsign(1) man page.

Mon, 8 May 2006 20:51



Top   Next   Prev   End

TCP wrapper support has been added to the rpcbind command. This support allows the administrator to limit calls to rpcbind to selected hosts. The administrator can also log all calls to rpcbind.

Tue, 9 May 2006 13:10



Top   Next   Prev   End

Partitioning: 8 Gb of memory = 16 Gb of swap, distributed around 12 drives = about 1.3 Gb/drive for each swap partition.

Recommended disk space for entire Solaris software group (everything except OEM stuff) is 6.7 Gb.

Tue, 9 May 2006 13:50



Top   Next   Prev   End

When running Oracle, might want to change the pagesize: http://learningsolaris.com/docs/Larger_page_sizes_bp.pdf

We can use multiple page-sizes: http://learningsolaris.com/docs/Multiple_page_sizes_bp.pdf

Memory intensive applications that have a large working set often perform suboptimally on the Solaris OS without a little tuning. This is because they make inefficient use of the microprocessor's translation lookup buffer (TLB) facility. MPSS allows you to exploit larger page sizes for the microprocessor's memory management unit (MMU, or M-Emu), which allows more efficient use of the TLB, ultimately resulting in improved application performance.

UltraSPARC supports four page sizes; 8 kilobyte (default), 64 kilobyte, 512 kilobyte, and 4 megabyte. To change:

    root# setenv LD_PRELOAD $LD_PRELOAD:mpss.so.1
    root# setenv MPSSHEAP 4M
    root# ./myprog
     
Tue, 9 May 2006 14:53



Top   Next   Prev   End

Good webpage for Solaris-10 freeware, like Webmin: http://www.sun.com/software/solaris/freeware/s10pkgs_download.xml

Got these from the docs.sun.com site:

    http://docs-pdf.sun.com/816-0281/816-0281.pdf
    http://docs-pdf.sun.com/816-4554/816-4554.pdf
    http://docs-pdf.sun.com/816-4555/816-4555.pdf
    http://docs-pdf.sun.com/816-4556/816-4556.pdf
    http://docs-pdf.sun.com/816-4557/816-4557.pdf
    http://docs-pdf.sun.com/816-4863/816-4863.pdf
    http://docs-pdf.sun.com/816-5041/816-5041.pdf
    http://docs-pdf.sun.com/816-5165/816-5165.pdf
    http://docs-pdf.sun.com/816-5166/816-5166.pdf
    http://docs-pdf.sun.com/816-5167/816-5167.pdf
    http://docs-pdf.sun.com/816-5168/816-5168.pdf
    http://docs-pdf.sun.com/816-5170/816-5170.pdf
    http://docs-pdf.sun.com/816-5171/816-5171.pdf
    http://docs-pdf.sun.com/816-5172/816-5172.pdf
    http://docs-pdf.sun.com/816-5173/816-5173.pdf
    http://docs-pdf.sun.com/816-5174/816-5174.pdf
    http://docs-pdf.sun.com/816-5175/816-5175.pdf
    http://docs-pdf.sun.com/816-5176/816-5176.pdf
    http://docs-pdf.sun.com/816-5177/816-5177.pdf
    http://docs-pdf.sun.com/817-0403/817-0403.pdf
    http://docs-pdf.sun.com/817-0404/817-0404.pdf
    http://docs-pdf.sun.com/817-0406/817-0406.pdf
    http://docs-pdf.sun.com/817-0544/817-0544.pdf
    http://docs-pdf.sun.com/817-0547/817-0547.pdf
    http://docs-pdf.sun.com/817-0552/817-0552.pdf
    http://docs-pdf.sun.com/817-1592/817-1592.pdf
    http://docs-pdf.sun.com/817-1975/817-1975.pdf
    http://docs-pdf.sun.com/817-1985/817-1985.pdf
    http://docs-pdf.sun.com/817-3000/817-3000.pdf
    http://docs-pdf.sun.com/817-3155/817-3155.pdf
    http://docs-pdf.sun.com/817-5093/817-5093.pdf
    http://docs-pdf.sun.com/817-5505/817-5505.pdf
    http://docs-pdf.sun.com/817-5506/817-5506.pdf
    http://docs-pdf.sun.com/817-6223/817-6223.pdf
    http://docs-pdf.sun.com/817-7519/817-7519.pdf
    http://docs-pdf.sun.com/819-0139/819-0139.pdf
     

Renamed:

    mv 816-0281.pdf 816-0281-font-admin-user-guide.pdf
    mv 816-4554.pdf 816-4554-sysadmin-IP-services.pdf
    mv 816-4555.pdf 816-4555-sysadmin-network-services.pdf
    mv 816-4556.pdf 816-4556-sysadmin-dns-nis-ldap.pdf
    mv 816-4557.pdf 816-4557-sysadmin-security-services.pdf
    mv 816-4863.pdf 816-4863-security-for-developers-guide.pdf
    mv 816-5041.pdf 816-5041-modular-debugger-guide.pdf
    mv 816-5165.pdf 816-5165-man1-user-commands.pdf
    mv 816-5166.pdf 816-5166-man1m-sysadmin-commands.pdf
    mv 816-5167.pdf 816-5167-man2-system-calls.pdf
    mv 816-5168.pdf 816-5168-man3-basic-functions.pdf
    mv 816-5170.pdf 816-5170-man3-networking-functions.pdf
    mv 816-5171.pdf 816-5171-man3-realtime-functions.pdf
    mv 816-5172.pdf 816-5172-man3-extended-functions.pdf
    mv 816-5173.pdf 816-5173-man3-interfaces-headers.pdf
    mv 816-5174.pdf 816-5174-man4-file-formats.pdf
    mv 816-5175.pdf 816-5175-man5-standards-env-macros.pdf
    mv 816-5176.pdf 816-5176-man6-demos.pdf
    mv 816-5177.pdf 816-5177-man7-device-network-interfaces.pdf
    mv 817-0403.pdf 817-0403-sysadmin-advanced.pdf
    mv 817-0404.pdf 817-0404-tunable-parameters.pdf
    mv 817-0406.pdf 817-0406-app-developer-guide.pdf
    mv 817-0544.pdf 817-0544-install-basic.pdf
    mv 817-0547.pdf 817-0547-whats-new.pdf
    mv 817-0552.pdf 817-0552-release-notes.pdf
    mv 817-1592.pdf 817-1592-sysadmin-containers-zones.pdf
    mv 817-1975.pdf 817-1975-resource-manager-guide.pdf
    mv 817-1985.pdf 817-1985-sysadmin-basic.pdf
    mv 817-3000.pdf 817-3000-sys-mgmt-agent-admin-guide.pdf
    mv 817-3155.pdf 817-3155-sys-mgmt-agent-developer-guide.pdf
    mv 817-5093.pdf 817-5093-sysadmin-file-systems.pdf
    mv 817-5505.pdf 817-5505-install-upgrade-planning.pdf
    mv 817-5506.pdf 817-5506-install-advanced.pdf
    mv 817-6223.pdf 817-6223-dynamic-tracing-guide.pdf
    mv 817-7519.pdf 817-7519-man3-multimedia-functions.pdf
    mv 819-0139.pdf 819-0139-fibre-channel-admin-guide.pdf
     
Tue, 16 May 2006 20:29



Top   Next   Prev   End
    Additional quickstart and developer documentation available at
       http://www.sun.com/bigadmin/content/selfheal/

    Solaris System Administration Guide
       http://docs.sun.com/app/docs/doc/817-1985

    Blogs:
       http://blogs.sun.com/sch
       http://blogs.sun.com/lianep
       http://blogs.sun.com/jwadams
       http://blogs.sun.com/dep
       http://blogs.sun.com/dminer
       http://blogs.sun.com/bobn
     
Thu, 15 Jun 2006 13:04



Top   Next   Prev   End

We may have to do some fiddling to get enough power for both the V890 and the current E450. Went to APC site, looked up V890 config:

    Server:                 Sun Sun Fire V890
    Total Power (Watts):    390
    # of Power Cords:       3
    Total Power (VA):       557
    Total Thermal (BTU/hr): 1332
    Plug Type:              NEMA L6-20P
    Operating Voltages:     200 or 208 or 230
     
Thu, 15 Jun 2006 17:27



Top   Next   Prev   End

Above assumed 4 drives; we're using 12, tower, UltraSparc IV chips.

First go to Large Corporations page: http://www.apc.com/solutions/index.cfm?segmentID=3

Click "Servers" on left menu, pick V890. We're using 2 CPUs, 12 drives, UltraSparc IV chips, and a tower server instead of rackmount. Results with those numbers:

    Server:                     Sun Sun Fire V890
        Total Power (Watts):    655
        # of Power Cords:       3
        Total Power (VA):       935
        Total Thermal (BTU/hr): 2237
        Operating Voltages:     120
    
    
    Recommendations:
    
        Best Price
        APC Smart-UPS SC 1500VA 120V - 2U Rackmount/Tower
        Price:  $389.99
        Runtime:  12 minutes
        Part Number(s): SC1500
        % of Max Capacity Used: 76%
    
        Best Value
        APC Smart-UPS 1500VA USB & Serial 120V
        Price:  $519.00
        Runtime:  14 minutes
        Part Number(s): SUA1500
        % of Max Capacity Used: 67%
    
        Best Performance
        APC Smart-UPS 2200VA USB & Serial 120V
        Price:  $839.00
        Runtime:  46 minutes
        Part Number(s): SUA2200
        % of Max Capacity Used: 33%
     
Thu, 22 Jun 2006 15:58



Top   Next   Prev   End

According to APC, Maximum power draw for the SUNFire V890 server fully populated is 16 amps at 208 volts or 3328VA. We should consider the following UPS solutions to protect the V890.

APC SmartUPS RT 5000, sku# SURT5000XLT ( NEMA L6-30 power plug - dedicated 30 amp breaker feed ) http://www.apc.com/resource/include/techspec_index.cfm?base_sku=SURT5000XLT

APC SmartUPS RT 7500, sku # SURT7500XLT ( no power plug, requires hardwire connection to dedicated 50 amp breaker panel ) http://www.apc.com/resource/include/techspec_index.cfm?base_sku=SURT7500XLT

SunFire V890 specifications (see POWER SUPPLY section) http://www.sun.com/servers/midrange/v890/specifications.jsp

Log entries | KWIC index | Other Logs