Wednesday, October 10, 2018

Updating inbound rules in security group using a lambda function

Prerequisites:

  1. Visual Studio 2017 with .NET core
  2. AWS Toolkit for Visual Studio 2017
  3. AWSSDK.Core tool kit
  4. AWSSDK.EC2 tool kit

Preparing the developer environment

  1. Visual Studio 2017 with .NET core :
    1. Log on to the microsoft downloads using the ensurance microsoft account.
    2. Download the profetinal version and installe VS2017 with C# development settings
  2. Installing AWS Toolkit for VS 2017 (Using Nuget):
    1. Open visual studio and go to Tools → Extentions and Updates
    2. In the Extentions and Updates dialog box, type "aws" in the search box. 
    1. Install the toolkit
  1. Creating a new solution for Lambda function:
    1. Create a blank solution in visual studio.
    2. Right click on te solution name and go to Add New Project menu
    3. Select AWS Lambda from the project list and select AWS Lambda Project with Tests (.NET Core) from the list.
    4. Two projects will be created with the given name and one project will be used as the test project.
    5. A new class will be created as Function.cs. Please done rename it.
  2. Add AWSSDKCore toolkit (Using nuget):
    1. Right click on the solution and click on Manage NuGet Packages
    2. Type AWS sdk core in the search box and select the AWSSDK Core tool kit
  3. Similarly, install the AWSSDK.EC2 using nuget packages.

Implementing the lambda function

  1. FunctionHandler is the core function in Function class. Any relevent proceedure should be executed within the handler as below.
  2. Refere AWS SDK online edition to findout code samples and requests used to sccess AWS objects
    1. https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/how-to-ec2.html
  3. Please see the code base below to undertand the implementation

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

using Amazon.Lambda.Core;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]

namespace UNOAutomationAWS
{
    public class Function
    {

        EC2Info e = new EC2Info();
        public string FunctionHandler(EC2Info e, ILambdaContext context)
        {
            return e.SetSecurityGroup();
        }
    }
}

using System;
using System.Collections.Generic;
using System.Text;
using Amazon.EC2;
using Amazon.EC2.Model;

namespace AutomationAWS
{
    public sealed class AmazonClient
    {
        private static AmazonClient _instance;
        private static readonly object simplelock = new object();
        private AmazonEC2Client _client = null;

        public static AmazonClient Instance
        {
            get
            {
                if (_instance == null)
                {
                    lock (simplelock)
                    {
                        if (_instance == null)
                        {
                            _instance = new AmazonClient();
                        }
                    }
                }
                return _instance;
            }
        }

        private AmazonClient()
        {
            _client = new AmazonEC2Client("<>", "<>", <>);
        }
       
        public AmazonEC2Client EC2Client
        {
            get
            {
                return _client;
            }
        }
    }

using System;
using System.Collections.Generic;
using System.Text;
using Amazon.EC2;
using Amazon.EC2.Model;
using System.Linq;

namespace AutomationAWS
{
    public class EC2Info
    {
       private const string SECURITYGROUPID = "<>";
        private const string INSTANCEID = "<>";
        private const string INSTANCENAME = "<>";

        private AmazonEC2Client _client;        

        public string SetSecurityGroup()
        {
            IpPermission ipPer = new IpPermission();
            _client = AmazonClient.Instance.EC2Client;

            //Get the EC2 Instance 
            var instance = GetInstance(INSTANCEID);

            if (instance != null && !instance.State.Name.Value.Equals("stopped"))
            {
                //Get the security group to modify with the new IP of EC2
                var securityGroup = GetSercurityGroup(SECURITYGROUPID);

                if (securityGroup != null)
                {
                    //Get IP permission to modify
                    var ipPermissions = securityGroup.IpPermissions.Where(i => i.Ipv4Ranges.Any(p => p.Description.Equals(INSTANCENAME)));

                    if (ipPermissions != null && ipPermissions.Count() > 0)
                    {
                        ipPer = ipPermissions.First();
                        var ip = ipPer.Ipv4Ranges.Where(p => p.Description.Equals(INSTANCENAME)).First().CidrIp;

                        //Remove the existing inbound rule
                        RemoveSecurityGroupRule(securityGroup, ipPer, INSTANCENAME, ip);
                    }                                       

                    //If no IpPermission found, create a new one.
                    if (ipPer.FromPort == 0)
                    {
                        ipPer.FromPort = 1521;
                        ipPer.ToPort = 1521;
                        ipPer.IpProtocol = "tcp";
                    }

                    //Set the new inbound rule in security group
                    SetSecurityGroupRule(securityGroup, ipPer, INSTANCENAME, instance.PublicIpAddress.Trim());
                }

                return instance.PublicIpAddress.ToString(); ;
            }

            return "";
        }      


        ///
        /// Get EC2 instance by instance id
        ///
        ///
        ///
        ///
        private Instance GetInstance(string securityGroupId)
        {
            var request = _client.DescribeInstancesAsync(new DescribeInstancesRequest()
            {
                InstanceIds = new List() { INSTANCEID }
            });

            if (request != null)
            {
                var result = request.Result;
                var reservation = result.Reservations.First();
                return reservation.Instances.First();
            }

            return null;
        }


        ///
        /// Get security group by security group ID
        ///
        ///
        ///
        ///
        private SecurityGroup GetSercurityGroup(string securityGroupId)
        {
            var request = _client.DescribeSecurityGroupsAsync(new DescribeSecurityGroupsRequest()
            {
                GroupIds = new List() { securityGroupId }
            });

            if (request != null)
            {
                var result = request.Result;
                return result.SecurityGroups.First();               
            }

            return null;
        }


        ///
        /// Removing the existing security rule
        ///
        ///
        ///
        ///
        private string RemoveSecurityGroupRule(SecurityGroup securityGroup, IpPermission ipPer, string name, string ip)
        {
            var ipr = new List { new IpRange() { CidrIp = ip, Description = name } };
            var request = _client.RevokeSecurityGroupIngressAsync(new RevokeSecurityGroupIngressRequest()
            {
                GroupId = securityGroup.GroupId,
                IpPermissions = new List
                {
                    new IpPermission()
                    {
                        FromPort = ipPer.FromPort,
                        ToPort = ipPer.ToPort,
                        IpProtocol = ipPer.IpProtocol,
                        Ipv4Ranges = ipr,
                        Ipv6Ranges = ipPer.Ipv6Ranges,
                        PrefixListIds = ipPer.PrefixListIds
                    }
                }
            });

            return request.Result.ToString();
        }


        ///
        /// Setting up the new security rule
        ///
        ///
        ///
        ///
        private string SetSecurityGroupRule(SecurityGroup securityGroup, IpPermission ipPer, string name, string ip)
        {
            var ipr = new List { new IpRange() { CidrIp = ip, Description = name } };
            var request = _client.AuthorizeSecurityGroupIngressAsync(new AuthorizeSecurityGroupIngressRequest()
            {
                GroupId = securityGroup.GroupId,
                IpPermissions = new List
                {
                    new IpPermission()
                    {
                        FromPort = ipPer.FromPort,
                        ToPort = ipPer.ToPort,
                        IpProtocol = ipPer.IpProtocol,
                        Ipv4Ranges = ipr,
                        Ipv6Ranges = ipPer.Ipv6Ranges,
                        PrefixListIds = ipPer.PrefixListIds

                    }
                }
            });

            return request.Result.ToString();
        }

    }
}



Wednesday, August 1, 2018


1.       [IMPORTANT] First of all, take a snapshot of the EBS volume for backup[1]
2.       Connect to your Windows instance and shrink the root volume from 1000 GB to 100 GB
a.       Open Disk Management on the Windows instance
b.      Right-click on volume C: -> Choose "Shrink Volume"
c.       Enter the amount of space to shrink in MB(in this case, 921600 MB)
3.       Stop the Windows instance
4.       Detach the root EBS volume from the Windows instance
5.       Launch an Amazon Linux instance in the same availability zone(AZ) with the Windows instance
a.       *I chose an EBS-optimised m4.large instance type to improve the disk speed[2]
6.       Attach the EBS volume to Amazon Linux instance as device /dev/sdf
7.       Create a new 100 GB EBS volume in the same AZ and attach it to the Amazon Linux instance as device /dev/sdg
8.       Connect to the Linux instance with ssh and copy all data from the original volume(/dev/sdf) to the new one(/dev/sdg)
a.       Type  "sudo -s" to switch to the root account
b.      Check that the volumes are attached correctly, type:
                                                               i.      fdisk -s /dev/sdf (the size of the 1000 GB volume should appear in KB, 1048576000 KB)
                                                             ii.      fdisk -s /dev/sdg (the size of the 100 GB volume should appear in KB, 104857600 KB)
c.       Type "yum install pv -y" to install pv. This tool will show you the progress of data transfer through a pipeline.
d.      Type "dd if=/dev/sdf |pv| dd of=/dev/sdg bs=1M"
                                                               i.      "dd" copy the contents of the source to the destination. The "bs=1M" tells "dd" to use a block size of one megabyte.
                                                             ii.      Please expect several hours to finish this step. In my environment, this step took five and half hours.
                                                            iii.      The time required depends on the instance type, network throughput, etc.
9.       Stop the Amazon Linux instance and detach both volumes(/dev/sdf and /dev/sdg)[4]
10.   Attach the 100 GB volume to the original Windows instance as device /dev/sda1 \
11.   Start the Windows instance
a.       If everything goes as expected, the Windows instance should pass the instance status checks.
b.      Logon to the Windows instance and verify from the Disk Management that the root EBS volume is 100 GB.
12.   Terminate Linux instance, delete the EBS volume of 1000 GB


Links:-

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.html
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-attaching-volume.html
http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ebs-detaching-volume.html

Shrink Database Log files (.ldf)

Open SQL Management Studio

Right Click on the database and go to Properties → Options

Select Recovery Model as "Simple". Click OK.
To Shrink Log file, Right click on the database go to Tasks → Shrink → Files
From the Shrink File window select Filegroup as "Log". Make sure to select "Release unused space" option. Click Ok.

Expand the root volume using the Amazon EC2 console

1.    Open the Amazon EC2 console and choose Instances from the navigation pane.

2.    Choose your instance, and choose the volume next to Block devices.

3.    Choose the EBS ID.
4.    Choose the volume, choose Actions, and select Modify Volume.

5.    Enter your desired volume in the Size field, choose Modify, and choose Yes.

6.    Refresh the console page. The expanded volume is immediately available for the OS. You can check this by choosing the Description tab, and choosing the information icon next to the State. This will show the Original Size and the Target Size.
Note: Windows root volumes are the master boot record (MBR) by default, and they can be extended up to 2TB. See Amazon EBS Volume Types for other considerations.

7.    Open the RDP session for the instance, launch the Disk Management view by using the diskmgmt.msc command, choose Action, and choose Refresh.

8.    Open the context (right-click) menu, and choose Extend Volume.

9.    Accept the default options by choosing Next and then Finish.

10.  Check the disk from This PC