Info Fast ZFS Storageserver with Oracle Solaris, OmniOS and napp-it

Toggle sidebar Toggle sidebar
G

gea

Senior member
Aug 3, 2014
210
12
81
With Oracle Solaris, the origin of ZFS and the most feature rich ZFS server
at the moment or OmniOS a free Solaris Fork you can build very fast and
stable storage systems with the best Windows compatibility on Unix
systems due the in ZFS embedded kernelbased SMB server from Sun/Oracle.

Up to now, there have been two weak points:
Especially with OSX, performance with SMB was low
Default settings are optimized for 1G networks

With the current Solaris 11.3 and the new OmniOS 151017+ SMB 2.1
is included with an enormous performance boost on OSX.

You need 10G if you want a performance comparable to a local disk
and you need 10G if you want to copy Terabytes in an acceptable time.
I have done some performance tests with tuning suggestions
that are optimized for fast 10G networks what gives me an SMB
performance on OSX and Windows up to 800-900MB/s.

My results with tuning options:
http://napp-it.org/doc/downloads/performance_smb2.pdf
 
  • Like
Reactions: igor_kavinski
Elixer

Elixer

Lifer
May 7, 2002
10,376
762
126
Conclusion
:
Use 10 GbE, SMB2+ and Jumboframes + some OS tunings if you need 10G performance
Care about OS version and drivers and about some driver settings like the irq throttling in X540 on Windows
Oracle Solaris is out of the box slightly faster than current OmniOS beta.
Both are a huge step forward compared to their predecessor especially on OSX or with 10 GbE.
For single large file and single user the sequential performance is ok with a pool from any sort of disks.
If you need a really high continuous write performace with many small files, you must use enterprise class SSDs
4-8 GB RAM is ok for a single user even with 10Gb. For multiuser or multi process access (videoediting for example
may read several streams in parallel), use as much RAM as possible. If you add an L2Arc enable sequential caching.
Click to expand...
Nice write up, thanks!
 
G

gea

Senior member
Aug 3, 2014
210
12
81
New OmniOS 151017 February beta with SMB 2.1 available
There is an update for OpenSSH that fix the sudo bug of the january beta.
Napp-it is working again with this update


update via:
pkg update


from OmniOS discuss:
Based on illumos-omnios master commit cc70d5b, uname -v reports "omnios-cc70d5b".


And omnios-build master commit 45f0d7b.


New with this update:


- Case-sensitivity now in ZFS test suite.


- GLDv3 improvement (by OmniTI's own Dale Ghent) which should produce less confusing dladm(1M) output.


- SunSSH and sudo now honor the illumos-bug-6057 ways of doing things. Please PLEASE look for other "last login" weirdness, but there shouldn't be any. If there is, PLEASE report it to this list ASAP.


- Speaking of SSH, OpenSSH now includes the full wad of Joyent work (modulo 1-2 small differences) to be more of a SunSSH replacement. If you haven't tried OpenSSH because you needed something from SunSSH, PLEASE try it now.




There is no new release media for this update, but I did update all of the packages, so you'll need to create a new BE and reboot.


Thanks,
Dan
Click to expand...
 
G

gea

Senior member
Aug 3, 2014
210
12
81
Info

OmniOS 151018 stable is available

Main advantage: many improvements regarding SMB2, see http://napp-it.org/doc/downloads/performance_smb2.pdf and ReleaseNotes/r151018

I am working on preconfigured images for a napp-it ZFS storage server
beginning with the SuperMicro Boards X11-SS where you cannot install OmniOS from USB sticks.

The ready to use images are cloned with CloneZilla. You can also use this method for backup and disaster recovery, see http://www.napp-it.org/doc/downloads/napp-it.pdf
 
G

gea

Senior member
Aug 3, 2014
210
12
81
Infos for the Napp-it ToGo ZFS Storage Server
I have uploaded new images (freely available)

New OmniOS 151018 stable with SMB2.1 and napp-it 16.02f,
Base-Tuning, TLS Alert Emails ex to Gmail. Alle Nics are autoconfigured for DHCP.


Napp-it ToGo Sata
These are images for a barebone/ hardware setup to an Sata disks.
After Cloning the image to Sata, your appliance is ready to use

The images are intended for a Kingston V300-60GB or an enterprise class
Intel S3510-80 with powerloss protection.
HowTo: http://www.napp-it.org/doc/downloads/napp-it.pdf

The images were created on an SuperMicro X11-SSH-CTF but are running on other mainboards and systems. If you have installed them successfully on other systems ex Dell or HP, please send an email to community@napp-it.org


Napp-it ToGo VM
These are images for a virtualized SAN/NAS under ESXi.
You can import the image ready to use (deploy a template)
HowTo: http://www.napp-it.org/doc/downloads/napp-in-one.pdf

Current Image 2016.04 for ESXi 5.5U2 - 6.0U2
Open-VM-Tools, vmxnet3s and MTU9000

Open-VM-Tools are now integrated in OmniOS 151018
 
G

gea

Senior member
Aug 3, 2014
210
12
81
OmniTi published drivers for the new 10G cards with Intel X550 chipset (2.5,5,10G) and drivers for the Intel X710 nic (single or duals 40G QSFP+ or up to 8 x 10G)


Drivers are in OmniOS 151019 bloody after a pkg update or downloadable

https://www.listbox.com/member/arch...6170139:826205AE-3405-11E6-B619-CE94D69137EA/


Integration in 151014 and 151018 is coming.
Good news for storage servers (barebone or AiO) with SuperMicro X10 SoC boards like my X10SDV-2C-7TP4F
 
G

gea

Senior member
Aug 3, 2014
210
12
81
Buginfo to all PoeF encrypted pool users

There is a bug ithat can lead to a non importable encrypted pool
Please update napp-it, thanks to Chris who discovered the bug

Napp-it.08 dev contains a new feature: Appliance Map
these are editable maps of all slots and the disk within with an option to add remarks, see http://www.napp-it.org/doc/downloads/napp-it.pdf chapter 22
 
G

gea

Senior member
Aug 3, 2014
210
12
81
Open-ZFS Developer Summit 2016

There are videos available at http://open-zfs.org/wiki/Main_Page
with infos about the state of

- ZFS native encryption (filesystem property not on underlying disks)
- faster Sequential Resilvering

They are in Oracle Solaris but Open-ZFS lacks these features at the moment.
Oracle Solaris is yet the fastest and most feature rich ZFS server but the difference will become lower.
 
Essence_of_War

Essence_of_War

Platinum Member
Feb 21, 2013
2,650
4
81
I'm really excited about the zfs level encryption. ThyVraiqFFAny8VkpZMpHP GitHub thread where Tom Caputi adds the commit was really cool!

Hopefully it can be merged into the Illumos/bsd/OS X versions also!
 
G

gea

Senior member
Aug 3, 2014
210
12
81
Development was done on Zol so there it will be available first.
Upstream is requested to Illumos and OSX.

I am quite sure BSD will follow to end incompatible encryption methods on Open-ZFS platforms.
 
G

gea

Senior member
Aug 3, 2014
210
12
81
info

OmniOS 151020 stable is out
https://omnios.omniti.com/wiki.php/ReleaseNotes/r151020

new.
NVMe 1.1 driver
Linux LX container support (from SmartOS, beta)

USB3 support for Illumos (OmniOS, OI, SmartOS) on the way
https://www.mail-archive.com/omnios-discuss@lists.omniti.com/msg07158.html

On updates, care about the move of Illumos from SunSSH to OpenSSH
https://omnios.omniti.com/wiki.php/Upgrade_to_r151020


napp-it 16.11.dev
From this release on I will remove support for older browsers to simplify the GUI code
first step: menu is css only with simple ul lists. (If you intend to modify the UI)
http://napp-it.org/downloads/changelog_en.html
 
G

gea

Senior member
Aug 3, 2014
210
12
81
Solaris being canned, at least 50% of teams to be RIF'd in short term - post regarding Oracle Corp. layoffs

Sun was the most innovative IT firm years ago with the best engineers and concepts.
Seems that many of their superiour ideas survive only because they were opensourced by Sun like ZFS, dtrace, the zones, the Linux container concept or servicemanagement and adopted by others. The Solaris OS with included projects like the ZFS integrated NFS/SMB/iSCSI services, Comstar or Crossbow may survive also only because the OpenSource fork Illumos as the successor of OpenSolaris that may become now the "real" Solaris.

Only rumours at the moment, but Oracle seems the worst what could happen to Sun and others who were bought by Oracle. More or less the Adobe project. Buy it and stop it as a possible competitor??
 
Last edited:
G

gea

Senior member
Aug 3, 2014
210
12
81
Info from OmniTi (OmniOS)

If you are user of LTS (r151014) or Stable (r151020), please "pkg update" and be ready for a new BE and a reboot.

Included in this update are:

- bge fixes on HP Gen9 systems
- MSI-X for NVMe is disable on VMware (where MSI-X is a problem)
- mmap() now properly modifies a file's timestamps in ZFS.
- Default NFS server threads are now increased.
- i40e now uses multiple receive and transmit rings
- One mpt_sas panic bugfix

The install media has been updated too, as the bge fix will help HP Gen9 users. Install media pointers can be found here:

https://omnios.omniti.com/wiki.php/Installation

and for OmniOS bloody:

Because I'm experimenting with Kayak-for-ISO, I want to have fresh illumos-omnios bits (esp. Loader itself) on the public server. I've just pushed new packages (full build) out to the bloody repo. Update highlights include:

- AIO improvements in LX
- Some loader bugfixes
- blkdev fixes for devices with blksize > 512 bytes.
- illumos 7777 "Expose xdf minor nodes when in PV-HVM mode" fix from Delphix. We are taking this pre-integration because of its potential to help the new Kayak-based ISO installer.

I will not be cutting traditional install media, given I'm getting closer on the Kayak-for-ISO front. Existing users should "pkg update". New users should either install the old bits and upgrade, or take their chances on the WIP Kayak-for-ISO installer.

Watch this space for an updated Kayak-for-ISO, which will at least have a controlling-terminal fix in it (so you can do job control on shells, AND invoke format(1M) w/o it complaining about /dev/tty).

Happy updating!
Dan
 
Last edited:
X

XavierMace

Diamond Member
Apr 20, 2013
4,307
450
126
Hey gea, not to hijack your thread too much but what's the best way to reach you these days with questions about issues I'm having? One of my two boxes has been having issues of late and I'm a bit stumped? You still more active at [H] and STH?
 
X

XavierMace

Diamond Member
Apr 20, 2013
4,307
450
126
Hey gea, is the main napp-it download broken right now? My existing box can pull updates but when I try to do a new install on a new box, it just gets a 38k download and does nothing. Using the normal wget -O - www.napp-it.org/nappit | perl
 
X

XavierMace

Diamond Member
Apr 20, 2013
4,307
450
126
Hey gea, thanks to replying both here and to my email. Chalk this one up to lack of sleep on my part. Thanks!
 
You must log in or register to reply here.

TRENDING THREADS

COMPANY
RESOURCES
FOLLOW
Top Bottom