Flexpod - Setting up a UCS Mini with Direct Attached Storage (DAS)

Overview
This entry will document what we did to setup a UCS Mini Flexpod with direct attached storage for a remote office. Using iSCSI.

This document will not cover step by step instructions on how to do common UCS or NetApp tasks(such as creating service profiles or a storage volume), only things specific to setting it up for using in the UCS Mini configuration.

Equipment
The hardware used:

Type	Model	Firmware / OS
Chassis	UCS 5108 AC2	3.0(1c)
Fabric Interconnects	UCS 6324	3.0(1c)
Blades	UCS B200 M3	3.0(1c)
Storage	FAS 2552 (no external storage)	Switchless cDOT 8.3

Reference
For information on how to setup a UCS Chassis from scratch, please see this awesome guide:

• Speak Virtual UCS Guide

I have used this guide to setup two, standard UCS Chassis as well as a framework for setting up the UCS Mini.  

Assumptions

I am assuming:

• The equipment is un-boxed and racked
• Do not turn anything on yet, just get it in the racks
• Proper network infrastructure exists (Cisco switches)
• I am not a network guy, so sadly this guide will not include any network configurations besides setting IP addresses.

Storage Configuration

The storage will be setup first.  My example is a FAS 2552 with dual controllers, without external storage in a switchless cDOT 8.3 cluster.

1. Download the official NetApp setup program and guide from the NOW site
• http://mysupport.netapp.com/NOW/public/system_setup/
• I am using the FAS 2552 setup guide
2. Plug in the power, but do not turn it on. Leave the power supply switch off.
3. Using the supplied network cable, connect the two ACP Ports. Has a picture of a wrench with a padlock on it.
4. Using SAS cables, connect the left SAS port on controller 1 into the right SAS port on controller 2.
5. Connect the right SAS port on controller 1 into the left SAS port on controller 2.
6. Using 10gig SFP cables, e0e on controller 1 to e0e on controller 2
7. Connect e0f on controller 1 to e0f on controller 2
8. Connect Remote Management port (wrench icon) to your management network.
9. Should look something like this:

• Forgive the cable mess, temporary setup before we shipped it to the remote site.
• Ports e0c and e0d are going to the Fabric Interconnects, we'll get to that later
10. Cable a windows workstation to the same network that the NetApp's management ports are connected to.
11. Install and run the NetApp_SystemStartup.exe
12. Press the Discover button
13. Turn on the controller power switches when directed
14. Follow the onscreen instructions to configure cluster information, authentication, disk, licenses, etc.
• Unfortunately, I did not take any screen shots during this part or take good notes like I should have.
15. If everything is cabled and configured correctly, the NetApp can be accessed over SSH or using OnCommand.
16. Create a SVM with iSCSI enabled

UCS Mini Configuration
Please follow the Speak Virtual UCS setup guide to get the UCS Mini mostly setup.  I am going to document what was different here. Also remember that this guide is configuring everything for use with iSCSI

First thing that was different was configuring the Unified Ports. On each Fabric Interconnect:

1. On LAN > Appliances > Fab a/b > VLANs
• Create a vlan. I called mine "iSCSI-App-A/B"
• Type in a vlan ID that is not in use on your network
• Select Fab A or B, not both
• Not native vlan
2. On Lan > Policies > Appliances > Network Control Policies
• Create a policy called "AppModeNCP"
• CDP Disabled
• MAC Register Mode: Only native vlan
• Action on Uplink Fail: Warning
3. On Equipment > Fab Interconnects > fab a/b > Select Configure Unified Ports
• Set ports 1 and 2 to Appliance Ports. 
• Set the VLAN to the VLAN created in step 1
• Set the access control policy to AppModeNCP
• The mode is Access
• Set ports 3 and 4 to Network port
• Mode is Trunk
4. Wait for the Interconnect to reboot. Then change the other Fabric Interconnect.
5. On LAN > Policies  > root > vNIC Templates
1. Create a vNIC Template for iSCSI_A
1. Select Fab A
2. Select Adapter
3. Select iSCSI-App-A for vlan
4. Updating Template
5. 9000 MTU
6. ISCSI_A mac pool (see the speak virtual doc on how to create a mac pool)
7. QoS Policy VMWare
8. Network Control Policy: AppModeNCP
9. Dyname vNIC
2. Create a vNIC Template for iSCSI_B
1. Select Fab B
2. Select Adapter
3. Select iSCSI-App-B for vlan
4. Updating Template
5. 9000 MTU
6. ISCSI_A mac pool (see the speak virtual doc on how to create a mac pool)
7. QoS Policy VMWare
8. Network Control Policy: AppModeNCP
9. Dyname vNIC

Next is cabling.

1. On Fab A
• Connect the top two ports to e0c on both NetApp controllers 1 and 2
• Connect bottom two ports to 10 gig ports on the network switches.
2. On Fab B
• Connect the top two ports to e0d on both NetApp controllers 1 and 2.
• Connect bottom two ports to 10 gig ports on the network switches.

There's some more stuff, I will update this page when I figure out what I did.