Netapp on BAFM

NetApp FAS/Data ONTAP public key authentification with CIFS/NFS license

Fri, 08 Aug 2014 08:29:59 +0000

Well as the title says, sadly we bought our FAS6210 without CIFS/NFS license. Thus, in order to create the folder structure/add the authorized_keys file, you’ll have to work for your money a little bit.

First, you need to run cifs setup / cifs passwd somewhere. I did it on our Data ONTAP simulator, which comes in handy for things like that.

You’ll get a cryptic looking password (no clue which format that is), looking like this: _OnWddr)xa.

NetApp - Copy LUN mappings

Fri, 08 Aug 2014 08:27:40 +0000

Well, today I had another idea (basically like the one I wrote for SVC’s VDisk mappings a while back) for a script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44


#!/bin/bash

KEY_FILE="/root/.ssh/netapp.dsa"
SSH_OPTS="/root/.ssh/netapp-ssh_config"

if [ $# -ne 3 ] ; then
 echo "fas-copy-lunmap: FAS_CONTROLLER SOURCE_IGROUP TARGET_IGROUP"
 echo
 echo "Copy the LUN map from one igroup to another, ie for ESX reinstallation"
 echo
 echo " Usage:"
 echo " - FAS_CONTROLLER: Hostname/IP-adress of the DATA ONTAP controller"
 echo " - SOURCE_IGROUP: igroup that is used as a reference for the copy process"
 echo " - TARGET_IGROUP: igroup that is actually modified"
 echo
 exit 1
fi

FAS_CTRL=$1
SOURCE=$2
TARGET=$3

ssh_fas() {
 # $@: commands for Data ONTAP
 COMMANDS="$@"
 /usr/bin/ssh -i $KEY_FILE -l root -F $SSH_OPTS $COMMANDS
}

#set -x

# Get the lun list.
for lun in $( ssh_fas $FAS_CTRL lun show -g $SOURCE | awk '{ print $1 }' | sort -u ); do

 # Get the LUN number its mapped
 LUN_ID="$( ssh_fas $FAS_CTRL lun show -m $lun | grep "^/vol" | awk '{ print $3 }' )"

 # If the LUN id is 0, skip otherwise we would copy the boot LUN
 if [ "$LUN_ID" != "0" ] ; then
 # Actually map the lun to our host
 echo "Mapping $lun to $TARGET as LUN_ID $LUN_ID"
 ssh_fas $FAS_CTRL lun map $lun $TARGET $LUN_ID
 fi
done
#set +x

I’ll post the counterpart of the script (to remove the LUNs) in a second post later on.

NetApp: Archive SnapManager SQL snapshots

Fri, 08 Aug 2014 08:25:56 +0000

As I wrote before, we’re using SnapManager (for SQL/Oracle) to create consistent snapshots. However my database guys don’t want to name their snapshots daily. (which I can understand), as once you archive those snapshots to a secondary (and tertiary) system, the names become junk.

So, they’re naming the snapshots like snap__vcsrv_29_12_2012-10.00.01. Sadly, when it comes to SnapVault, it expects the names in form of daily. otherwise you won’t be able to transfer the snapshots with the CLI (none that I have found anyway).

But we didn’t want to move away from naming the snapshots the way they are, so I ended up writing a PowerShell script, that once triggered archives the Snapshots needed for a set of databases. It took me a while to figure a bunch of stuff out, but in the end I think I have a working way of archiving custom-named snapshots.

Implementing SnapVault backups - the hard way

Fri, 08 Aug 2014 08:25:10 +0000

Well, I recently had the pleasant task of implementing SnapVault backups, that are being shipped to an offsite location with SnapMirror.

That in itself isn’t the bad thing, however we decided against Protection Manager (since it was a charged product back when we decided on this). So I basically had the three tasks:

Actually implement the SnapVault stuff (and learn my way around it and also document it)
Write a bunch of scripts, that help us in creating scheduled backups of our databases
Create a monitoring script, that’ll fit into our Nagios environment already in place

Well, two months later (sadly it still has some kinks - I can’t figure out this one bug though for the life of it) and a few hundred hours of working on/with it and out came four things:

NetApp: Establishing SnapVault relations

Fri, 08 Aug 2014 08:22:46 +0000

I’ve been spending a lot of my time the last week on getting SnapVault with out FAS-filers to work. Out came a script, which does this for a given volume (and of course SnapVault Primary and Secondary).

The script expects, that SSH public key authentification has been set up.

NetApp: Monitoring of SnapVault/SnapMirror/LUN/Snapshot information with Nagios

Sun, 15 Dec 2013 15:51:03 +0000

As I wrote before, we have a bunch of filers (and a ton of volumes w/ luns on them), that I need to monitor. At first, I tried the existing NetApp Nagios-Plugin(s), but they all use SNMP and with that I can either watch all volumes or none. And that didn’t satisfy me.

Don’t get me wrong, the existing plugins are okay and I still use them for stuff (like GLOBALSTATUS or FAN/CPU/POWER) which isn’t present in the API or real hard to get at, however I wanted more. So I ended up looking at the NetApp API, and ended up writing a “short” plugin for Nagios using Perl.

Maybe if I’m ever bored, I’ll rewrite it using C, but for now the Perl plugin has to suffice.

So far the plugin supports the following things:

Monitoring FlexVolumes (simply watching the free space)
Monitoring LUN space (the allocated space inside a FlexVolume for iSCSI/FC LUNs)
Monitoring Snapshot space (the allocated space inside a FlexVolume for Snapshots)
Monitoring SnapVault relations (and their age)
Monitoring SnapMirror relations (and their age)

The plugin will return performance data for most (if not all) of those classes. It needs a user on the filer you wish to monitor - which sadly needs to have the admin role.

XenServer 6-0-2: Fixing Root-Disk-Multipathing with Boot-from-SAN

Tue, 16 Jul 2013 16:27:12 +0000

As the title pretty much tells, I’ve been working on fixing the Root-Disk-Multipathing feature of our XenServer installations. Our XenServer boot from a HA-enabled NetApp controller, however we recently noticed that during a controller fail-over some, if not all, paths would go offline and never come back. If you do a cf takeover and cf giveback in short succession, you’ll end up with a XenServer host that is unusable, as the Root-Disk would be pretty much non-responsive.

Dealing with SnapVault replication issues

Tue, 09 Apr 2013 16:05:30 +0000

Well, for the past two months I had a case open with NetApp to figure out this SnapVault replication issue we were seeing. The initial transfer of the SnapVault relation would complete with a hick up, manual snapshot transfers also work - just the scheduled, auto-created Snapshots won’t replicate.

At first I (and the NetApp support) thought this was an issue with SnapVault itself, however after being away for the last four weeks I looked at the issue with fresh eyes. After a short peek into the logs, I found what I had found back when I first looked into this.

Generate Nagios config for check_netapp-api-pl

Tue, 05 Feb 2013 19:46:24 +0000

As so often, I wanted a script, that’ll crawl my filers and regenerate the configuration if there are any new volumes/snapvaults/snapmirrors or if one of them has been removed.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96


#!/bin/bash

FAS_HOSTS="$( ls /etc/nagios/objects/hosts/san/fas*{a,b}.cfg | cut -d/ -f7 | cut -d. -f1 )"

for host in $FAS_HOSTS; do
 OUTPUT_FILE=/etc/nagios/objects/hosts/san/$host-vol.cfg

 # Clear the output file
 echo "" > $OUTPUT_FILE

 # Get the volume list
 for volume in `ssh $host vol status | awk '{ print $1 }' | grep ^vol | sort -u | grep -v vol0$`; do
 user="$( grep "USER=" /etc/netapp-sdk/$host | cut -d= -f2 )"
 pass="$( grep "PASS=" /etc/netapp-sdk/$host | cut -d= -f2 )"
# echo "define service {"
# echo " use generic-service"
# echo ""
# echo " check_command check_netapp-volfree!$user!$pass!${volume}!92!98"
# echo " check_interval 5"
# echo " host_name ${host}"
# echo " notifications_enabled 0"
# echo " notification_interval 720"
# echo " service_description VOLSPACE ${volume}"
# echo "}"
 echo
 echo "define service {"
 echo " use generic-service-san-perfdata"
 echo ""
 echo " check_command check_netapp-lunspace!$user!$pass!${volume}"
 echo " check_interval 5"
 echo " host_name ${host}"
 echo " notifications_enabled 0"
 echo " notification_interval 720"
 echo " service_description LUNSPACE ${volume}"
 echo "}"
 echo

 SR="$( ssh $host snap reserve $volume | cut -d -f7 )"
 if [ "$SR" != "0%" ] ; then
 echo "define service {"
 echo " use generic-service-san-perfdata"
 echo ""
 echo " check_command check_netapp-snapreserve!$user!$pass!${volume}"
 echo " check_interval 10"
 echo " host_name ${host}"
 echo " notifications_enabled 0"
 echo " notification_interval 720"
 echo " # SR: $SR"
 echo " service_description SNAPRESERVE ${volume}"
 echo "}"
 echo
 fi
 done | tee -a $OUTPUT_FILE

 # Check snapvault foo
 for sv in `ssh $host snapvault status -l 2>/dev/null | awk '{ print $2 }' | grep vol`; do
 # only do the checks on sv_secondary
 if [ "$( echo $sv | grep $host | cut -d: -f1 )" == "${host}" ]; then
 vol="$( echo $sv | cut -d/ -f3 )"
 user="$( grep "USER=" /etc/netapp-sdk/$host | cut -d= -f2 )"
 pass="$( grep "PASS=" /etc/netapp-sdk/$host | cut -d= -f2 )"
 echo "define service {"
 echo " use generic-service-san-perfdata"
 echo ""
 echo " check_command check_netapp-snapvault!$user!$pass!$vol!38!42!"
 echo " check_interval 60"
 echo " host_name ${host}"
 echo " notifications_enabled 0"
 echo " notification_interval 720"
 echo " service_description SNAPVAULT ${vol}"
 echo "}"
 echo
 fi
 done | tee -a $OUTPUT_FILE

 # Check snapmirror foo
 for sm in `ssh $host snapmirror status 2>/dev/null | awk '{ print $2 }' | grep vol | grep $host`; do
 # only do the checks on sm_secondary
 if [ "$( echo $sm | grep $host | cut -d: -f1 )" == "${host}" ]; then
 vol="$( echo $sm | cut -d/ -f3 | cut -d: -f2 )"
 user="$( grep "USER=" /etc/netapp-sdk/$host | cut -d= -f2 )"
 pass="$( grep "PASS=" /etc/netapp-sdk/$host | cut -d= -f2 )"
 echo "define service {"
 echo " use generic-service-san-perfdata"
 echo ""
 echo " check_command check_netapp-snapmirror!$user!$pass!$vol!38!42!"
 echo " check_interval 60"
 echo " host_name ${host}"
 echo " notifications_enabled 0"
 echo " notification_interval 720"
 echo " service_description SNAPMIRROR ${vol}"
 echo "}"
 echo
 fi
 done | tee -a $OUTPUT_FILE
done

Generate Nagios config for NetApp filers

Tue, 01 Jan 2013 09:46:39 +0000

At some point in the last few weeks, I repeatedly had to recreate my Nagios config for currently six filers. After doing that a few times, I ended up (like sooo often) writing a short Bash script, that’ll do this for me - without any fuss.

The only thing the script needs, is that the filers and the filers are registered in DNS … Here’s an example:

1
2
3
4


fas3240a IN A 172.31.76.150
fas3240a-sp IN A 172.31.74.150
fas3240b IN A 172.31.76.151
fas3240b-sp IN A 172.31.74.151

With that done, the script will create the necessary Nagios config for those filers.

NetApp: Establishing SnapMirror relationships

Mon, 31 Dec 2012 09:44:19 +0000

After figuring out the SnapVault stuff, I needed to implement a whole bunch of SnapMirror relations. As I am lazy (as in click-lazy), I ended up writing a somewhat short Bash script, that’ll either establish a bunch of SnapMirror relations (for a single host) or just for a single volume.

The script expects, that SSH public key authentification has been set up, and that the source for the SnapMirror exists and is online/not-restricted.

NetApp: SnapVault snapshot retention for non-standard snapshot names

Sat, 29 Dec 2012 09:37:48 +0000

Well, the name says it pretty much. Once you rename the snapshot on the SnapVault destination from daily.0 to something else, the whole builtin SnapVault snapshot retention isn’t gonna work anymore.

Back when I started all the code-writing, I wasn’t aware of this. One of my co-worker complained to me about it on Wednesday that there are an assfull of snapshots on the SnapVault destination (one snapshot each day since the end of October, meaning more than 50 snapshots per volume, in a total of 12 or so FlexVolumes, making the total about 500 snapshots).

So I took the time to write this little Bash script (yeah, I know I’m mixing a bunch of languages - I really like the KISS principle), which will get the necessary information from the filer (snapvault snap sched needs to be set) and then deletes the over-aged snapshots.

NetApp: Archive SnapManager SQL Snapinfo

Sat, 29 Dec 2012 09:16:45 +0000

The MSSQL admins decided to dump the SMSQL Snapinfo stuff on a separate volume, that SMSQL also snapshots. Same as before, I need a PowerShell script that’ll archive the snapshot and rename it.

NetApp: Archive SnapManager Oracle Snapshots

Sat, 29 Dec 2012 09:12:57 +0000

And here’s the script for SMO. However, since different people administrate the Oracle Databases, they don’t want me to tinker with the database like the MSSQL admins. They give me a CSV-list of volumes, that should be backed up and I work with that.

vCenter: Removing VSC custom attributes

Wed, 17 Oct 2012 20:10:01 +0000

Well, yesterday I got pissed of those Virtual Storage Console custom attributes.

Currently we don’t use the Provisioning & Cloning feature of the VSC, thus we don’t need the custom attributes. After poking around, I decided to write a short PowerCLI script to do the task.

It’s really rather simple, so here goes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


param( [string] $vCenter )

# Add the VI-Snapin if it isn't loaded already
if ( (Get-PSSnapin -Name "VMware.VimAutomation.Core" -ErrorAction SilentlyContinue) -eq $null )
{
 Add-PSSnapin -Name "VMware.VimAutomation.Core"
}

If ( !($vCenter) )
{
 Write-Host
 Write-Host "vcenter-remove-vsc-attributes: <vcenter-server>"
 Write-Host
 Write-Host " <vcenter-server> - DNS name of your vCenter server."
 Write-Host
 exit 1
}

Connect-VIServer -Server $vCenter

Remove-CustomAttribute -CustomAttribute "PnC.CustSpec", "PnC.Deployed", "PnC.GroupID", "PnC.Source" -Confirm:$false

Disconnect-VIServer -server $vCenter -Confirm:$false

NetApp: Changing DS4243 shelf ID

Thu, 11 Oct 2012 21:17:39 +0000

I’m working on a project right now, providing a SnapVault target for our “big” NetApp. So we moved our 3240 to it’s target location, I spent most of my time yesterday doing the cabling (SAS and ACP, as well as power).

I’m still not finished, I still need to " beautify" the power cables, need to fix the network cables (currently I don’t have none ….) and some other minor stuff. But lemme skip back a bit.

NetApp: Migrating FCP luns with ndmpcopy to another controller

Sat, 06 Oct 2012 06:31:10 +0000

Well, I’m in a situation, where I need to move all volumes from one controller to two others. So I looked at the ways I had available:

Freshly implementing everything: No option at all!
vol copy: Is rather slow, thus no option
ndmpcopy: That’s exactly what I needed!

ndmpcopy is a great way to copy over a whole volume including it’s files (thus FCP luns) to another volume/controller.

First I threw in a crossover cable, since at around 6 PM our backup system starts it’s daily run, and everything else running via IP in between 6 PM and 6 AM is seriously impaired by this. Configured the additional ports on all three controllers (picked a private, not-routed range just in case) and then kicked of a simple bash script that ran the following:

NetApp LUN creation/vol sizing

Thu, 20 Sep 2012 04:49:43 +0000

Well, as you might know I’ve been tinkering with a NetApp FAS at work. The last few months, I’ve been trying to figure out a few things, which I actually did.

One “error” I ran into with creating the lun’s and volumes by hand was that the volumes were running out of space. Even if the volume was a bit larger than the LUN. After that happened a few times, I decided to see how to fix that. As it turns out, the GUI “fixes” that already in a way I wouldn’t have expected.

SnapManager for SQL Server: Service fails to respond in a timely fashin

Thu, 30 Aug 2012 17:34:58 +0000

Well, we recently upgraded the SnapManager version on our test box to 6.4.1. Now however, after restarting the box the SnapManager service failed to start … The error was something like this:

Now, first I stumbled upon this NetApp Community post, which only contained the “solution” to increase the global! wait time for services. That didn’t sit well with me.

So after looking through NOW! for a bit, I actually found the correct way. The fix is described in KB2010835. Yet again, another certificate error. Why do vendors deploy SSL certificates, when they use untrusted ones, which defeats the purpose of SSL certs or at least “brings up” users to ignore any error message they get concerning SSL certificates ?

NetApp - Remove LUN mappings

Fri, 15 Jun 2012 14:30:58 +0000

As promised in the earlier post, for completeness sake, here’s the counterpart for removing the LUNs in the first place.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


 #!/bin/bash

KEY_FILE="/root/.ssh/netapp.dsa"
SSH_OPTS="/root/.ssh/netapp-ssh_config"

if [ $# -ne 2 ] ; then
 echo "fas-remove-lunmap: FAS_CONTROLLER TARGET_IGROUP"
 echo
 echo "Remove every mapped lun from the target igroup (for example reinstallation)"
 echo
 echo " Usage:"
 echo " - FAS_CONTROLLER: Hostname/IP-adress of the DATA ONTAP controller"
 echo " - TARGET_IGROUP: igroup that is actually modified"
 echo
 exit 1
fi

FAS_CTRL=$1
TARGET=$2

ssh_fas() {
 # $@: commands for Data ONTAP
 COMMANDS="$@"
 /usr/bin/ssh -i $KEY_FILE -l root -F $SSH_OPTS $COMMANDS
}

# Get the hostname of the controller, necessary for the reporting
CTRL_HOSTNAME="$( ssh_fas $FAS_CTRL rdfile /etc/rc | grep ^hostname | cut -d -f2 | tr 'a-z' 'A-Z' )"

#set -x
# Get the lun list.
for lun in $( ssh_fas $FAS_CTRL lun show -g $SOURCE | awk '{ print $1 }' | sort -u ); do

 # If the LUN id is 0, skip otherwise we would remove the boot LUN
 if [ "$LUN_ID" != "0" ] ; then
 # Actually remove the mapping to our host
 echo "Removing $lun from $TARGET"
 ssh_fas $FAS_CTRL lun unmap $lun $TARGET
 fi
done
#set +x

With that, you can simply run it against a NetApp controller and remove every LUN map except the one with LUN ID 0 (which is pretty handy when installing/reinstalling ESX servers).

NetApp - Get a list of volumes containing too much LUNs

Fri, 08 Jun 2012 19:04:53 +0000

Well, after figuring things out (and realizing that if you create a LUN in the same size as the volume it’ll break), I decided to write yet another script to figure out which LUNs needed fixing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62


#!/bin/bash

KEY_FILE="/root/.ssh/netapp.dsa"
SSH_OPTS="/root/.ssh/netapp-ssh_config"
FAS_CTRL=$1

ssh_fas() {
 # $@: commands for Data ONTAP
 COMMANDS="$@"
 /usr/bin/ssh -i $KEY_FILE -l root -F $SSH_OPTS $COMMANDS
}

for volume in $( ssh_fas $FAS_CTRL vol status | grep "vol.* online" | awk '{ print $1 }' ); do
 VOL_SIZE="$( ssh_fas $FAS_CTRL vol size $volume | awk '{ print $8 }' | sed "s,.,," )"
 VOL_UNIT_PARAM="$( echo "${#VOL_SIZE} - 1" | bc )"
 VOL_UNIT="${VOL_SIZE:$VOL_UNIT_PARAM:1}"

 case $VOL_UNIT in
 k) CONV="1024";;
 m) CONV="1024 * 1024";;
 g) CONV="1024 * 1024 * 1024";;
 esac

 VOL_SIZE="$( echo "scale=0; ${VOL_SIZE/$VOL_UNIT/} * $CONV" | bc )"

 # Subtract the snap reserve (if any)
 SNAP="$( ssh_fas $FAS_CTRL snap reserve $volume | cut -d -f7 | sed "s,%,," )"

 if [ $SNAP -gt 0 ] ; then
 SNAP_SIZE="$( echo "$VOL_SIZE * $SNAP / 100" | bc )"
 REMAINING_VOL_SIZE="$( echo "$VOL_SIZE - $SNAP_SIZE" | bc )"
 else
 SNAP_SIZE=0
 REMAINING_VOL_SIZE=$VOL_SIZE
 fi

 # Get the luns created inside the volume
 LUN_SIZE_TOTAL=0
 for lunsize in $( ssh_fas $FAS_CTRL lun show -l ${volume} | awk '{ print $3 }' | sed -e "s,(,," -e "s,),," ); do
 LUN_SIZE_TOTAL="$( echo "$LUN_SIZE_TOTAL + $lunsize" | bc )"
 REMAINING_VOL_SIZE="$( echo "$REMAINING_VOL_SIZE - $lunsize" | bc )"
 done

 # Get the estimated space necessary.
 VOL_RESERVE="$( echo "$VOL_SIZE * 3 / 100" | bc )"
 echo " VOL_NAME: $volume"
 echo " VOLUME_SIZE: $( echo "scale=2; $VOL_SIZE / 1024 / 1024 / 1024" | bc )G"
 echo " - SNAP_SIZE: $( echo "scale=2; $SNAP_SIZE / 1024 / 1024 / 1024" | bc )G"
 echo " SUB_TOTAL: $( echo "scale=2; ($VOL_SIZE - $SNAP_SIZE) / 1024 / 1024 / 1024" | bc )G"
 echo
 echo " - TOTAL_LUN_SIZE: $( echo "scale=2; $LUN_SIZE_TOTAL / 1024 / 1024 / 1024" | bc )G"
 echo " SUB_TOTAL: $( echo "scale=2; ($VOL_SIZE - $SNAP_SIZE - $LUN_SIZE_TOTAL) / 1024 / 1024 / 1024" | bc )G"
 echo
 echo " MISSING_SPACE:"
 DIFFERENCE_VOL_LUN="$( echo "scale=2; ($VOL_SIZE - $SNAP_SIZE - $LUN_SIZE_TOTAL)" | bc | sed "s,-,," )"
 echo " DIFFERENCE_VOL_LUN: $( echo "scale=2; $DIFFERENCE_VOL_LUN / 1024 / 1024 / 1024" | bc )G"
 echo " + VOL_RESERVE: $( echo "scale=2; $VOL_RESERVE / 1024 / 1024 / 1024" | bc )G"
 echo " SUB_TOTAL: $( echo "scale=2; ($DIFFERENCE_VOL_LUN + $VOL_RESERVE) / 1024 / 1024 / 1024" | bc )G"
 echo
 echo
 echo
done

And with that you have a list of volumes, with the amount of space they need to resized in order to accomodate the contained LUNs and the snapshots.

NetApp - Get a list of all volumes not being used

Wed, 30 May 2012 17:40:12 +0000

Well, I had another task for today … I have an amount of FlexVolumes (sixty currently per controller), and I didn’t know if we had any, that didn’t have any LUNs on them. Now I thought there was a command for that since my co-worker mentioned something like that. However, once again … there isn’t.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39


#!/bin/bash

MAILTO="san@barfoo.org"
KEY_FILE="/root/.ssh/netapp.dsa"
SSH_OPTS="/root/.ssh/netapp-ssh_config"
FAS_CTRL=$1
TMPDIR="$( mktemp -d )"

ssh_fas() {
 # $@: commands for Data ONTAP
 COMMANDS="$@"
 /usr/bin/ssh -i $KEY_FILE -l root -F $SSH_OPTS $COMMANDS
}

# Get the hostname of the controller, necessary for the reporting
CTRL_HOSTNAME="$( ssh_fas $FAS_CTRL rdfile /etc/rc | grep ^hostname |
 cut -d -f2 | tr 'a-z' 'A-Z' )"

# Get a list of all volumes / luns
VOL_LIST="$( ssh_fas $FAS_CTRL vol status | grep ^vol |
 egrep -v '(nfs|cifs)' | awk '{ print $1 }' )"
LUN_LIST="$( ssh_fas $FAS_CTRL lun show | grep '/vol' |
 awk '{ print $1 }' )"

for lun in $LUN_LIST; do
 VOL_EXTRACT="$( echo $lun | cut -d/ -f3 )"
 VOL_LIST=${VOL_LIST/${VOL_EXTRACT}/}
done

for vol in $VOL_LIST; do
 echo "Empty Flex Volume: $vol."
done > $TMPDIR/mailcontent

if [ "$( grep Flex $TMPDIR/mailcontent )" ] ; then
 cat $TMPDIR/mailcontent | mailx -r $MAILTO
 -s "$CTRL_HOSTNAME: Empty volume check" $MAILTO
fi

rm -r $TMPDIR

Automating qual_devices updates

Wed, 30 May 2012 17:23:58 +0000

Well, once again I was presented with a nice AutoSupport warning once I logged into my NOW account. Since we don’t have CIFS and/or NFS licensed on our filers, I wrote a cute little script that does the whole work for me.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88


#!/bin/bash

TMPDIR="$( mktemp -d )"
KEY_FILE="/root/.ssh/netapp.dsa"
SSH_OPTS="/root/.ssh/netapp-ssh_config"
FAS_CTRL=$1
QUALDEVICES=$2

ssh_fas() {
 # $@: commands for Data ONTAP
 COMMANDS="$@"
 /usr/bin/ssh -i $KEY_FILE -l root -F $SSH_OPTS $COMMANDS
}
#set -x

echo "Updating qual_devices on $FAS_CTRL"

# Enable ftpd if it isn't enabled already
echo -n "Checking FTP subsystem"

FTPD_INITIAL_STATE="$( ssh_fas $FAS_CTRL options ftpd.enable |
 awk '{ print $2 }' )"
if [ $FTPD_INITIAL_STATE == "off" ] ; then
 ssh_fas $FAS_CTRL options ftpd.enable on
 echo " .... ok"
else
 echo " .... ok"
fi

echo
read -s -p "Please supply the root password for $FAS_CTRL: "
ROOT_PASSWD=$REPLY

echo
echo "Checking qual_devices:"
echo -n " Running: "
mkdir $TMPDIR/fas-version
# Check the old version.
ftp -n $FAS_CTRL >/dev/null <<END_SCRIPT
prompt
user root $ROOT_PASSWD
lcd $TMPDIR/fas-version
cd /etc
mget qual_devices*
quit
END_SCRIPT

FAS_VERSION="$( grep Datecode $TMPDIR/fas-version/qual_devices_v3 |
 head -n1 | cut -d -f3 )"

echo "$FAS_VERSION"
# Unzip the qual_devices.zip file and compare it.
ORIGPWD=$PWD
mkdir $TMPDIR/new-version
cd $TMPDIR/new-version
unzip $QUALDEVICES >/dev/null
echo -n " New: "
NEW_VERSION="$( grep Datecode $TMPDIR/new-version/qual_devices_v3 |
 head -n1 | cut -d -f3 )"
echo "$NEW_VERSION"
echo

# Upload the supplied version if the new file doesn't match the one running
# on the controller
if [ $NEW_VERSION != $FAS_VERSION ] ; then
 echo "Uploading qual_devices to $FAS_CTRL"
 ftp -n $FAS_CTRL >/dev/null <<END_SCRIPT
prompt
user root $ROOT_PASSWD
lcd $TMPDIR/new-version
cd /etc
mput qual_devices*
quit
END_SCRIPT

 # Send an asup message, that the issue is corrected
 echo "Generating AutoSupport message"
 ssh_fas $FAS_CTRL options autosupport.doit qual_devices_fixed_$NEW_VERSION
fi

#set +x

# Enable ftpd
if [ $FTPD_INITIAL_STATE == "off" ] ; then
 ssh_fas $FAS_CTRL options ftpd.enable off
fi

rm -r $TMPDIR

The whole thing is surly based, that FTP is configured ( as I described previously).

TSM and NetApp - Another Quick Hint

Sun, 27 May 2012 13:45:22 +0000

Well, we’ve been trying to come up with a decent way to backup NetApp snapshots to tape (SnapMirror To Tape), so we evaluated all the available methods of using NDMP backups.

There’s Image Backup in two different variants - FULL and DIFFerntial
There’s SnapMirror To Tape

So the Image Backup is one of the ways. However the DIFFerntial backup only works for CIFS and NFS shares (which we don’t use). We only have FC luns (or rather FCoE luns), so there’s only a single (or in case of the boot luns more than one) file in each volume. With that however, each run of the Image Backup with the DIFFerential option, it’s gonna backup the full size of the volume (plus the deduplicated amount).

TSM and NetApp - Quick Hint

Wed, 25 Apr 2012 20:09:24 +0000

Well, to save everyone else the trouble (since it isn’t documented anywhere - and I just spent about an hour finding the cause for this), if you need to configure NDMP on your NetApp Filer, make sure you also configure an interface other than e0M.

Apparently the necessary controlport for NDMP (10000) is being blocked on e0M, thus ndmp may be configured and running, however TSM is gonna complain that it is unable to connect to the specified data mover.