Azure, Architecture, Automation
Sometimes, the need for a certificate to verify something quickly. Here is an example how to add certificate that expire after 50 years on two domain names from your Powershell prompt.
New-SelfSignedCertificate -DnsName "app.domain.local", "test-app.domain.local" -CertStoreLocation "cert:\LocalMachine\My" -NotAfter (Get-date).AddYears(50) -KeyExportPolicy ExportableA decision to make when developing a hybrid cloud, or just providing access to Azure (or might as well be AWS or GCP) is if a VPN connection will suffice, or you will need an dedicated circuit like Express Route (AWS – Direct Connect, GCP- Dedicated Internetconnect).
Looking on the Microsoft documentation on ExpressRoute, they promise a 99.9 % SLA uptime on the connection. A VPN connection is no SLA on. This is due to Microsoft provisioning redudant circuits to the provider edge in the ExpressRoute scenario and thus can give an SLA.
Be aware to make sure the provider match the SLA to your customer edge. This may differ.
There is a lot to be said about what and what not you can achieve with ExpressRoute or VPN (or combining). To start somewhere, a simple test were conducted. Two similiar Azure environments were setup, one with ExpressRoute and one with VPN – to the same on-premise datacentre.
The test is one ping message, sent every 5 seconds to each Azure environments over 24 hours. The tests were done at the same time. What we want out of the tests are two things, what is the delay and do we have any packet drops?
This were the results
So what is the differences?
Summary
The biggest advantage of ExpressRoute seems to mitigate the worst case scenarios and more predictable response time, as advertised by Microsoft. Average Latency wise there is also an advantage of ExpressRoute, however, seen in pure ms, not too much of a difference, depending on your application needs.
API Management makes it possible to publish APIs for exteranl and/or internal purposes in a secure manner. The Developer or Premium tier enables virtual network support, giving the possibility to access resources in your virtual network, or on-premise if you have VPN/ExpressRoute setup.
There is two different kind of network integrations for API management, external and internal. The External are external publishing and the gateway are able to access internal resources.
While an Internal only offer it API services internally
For our example, we will go with the External.
Before proceeding, a dedicated subnet need to be setup just for API management (a /29 works fine) and if there is an advertised default route, there is a need to add a User defined route to Internet as a default one.
First off, we need the ResourceID for the subnet, copy paste the resourceID
$VNET = Get-AzureRmVirtualNetwork -Name "API-VNET" -ResourceGroup "Common-RG"
Get-AzureRmVirtualNetworkSubnetConfig -Name "API-VNET-SUBNET" -VirtualNetwork $VNETAfter this, defined the API Management Network and save it to the variable $apivnet
$Location = "WestEurope"
$apivnet = New-AzureRmApiManagementVirtualNetwork -Location $Location -SubnetResourceId "/subscriptions/b6c5a626-dfd6-40ca-9408-525e7345e8de/resourceGroups/Common-RG/providers/Microsoft.Network/virtualNetworks/API-VNET3/subnets/API-VNET-SUBNET"Now, create the API Management, specify a Name, an Organization, AdminEmail and the SKU (need to be Developer or Premium) and finally the VpnType, that is the networking type.
$APIRG = New-AzureRMResourceGroup -Name "API-RG" -Location $Location
New-AzureRmApiManagement -ResourceGroupName $($APIRG.ResourceGroupName) -Name "api-dev" -Location $Location -Organization "SystembolagetDev" -AdminEmail "foo@bar.org" -Sku Developer -VirtualNetwork $apivnet -VpnType ExternalThe command will take a while, since the VNET integration takes up to 15 minutes to complete. After that, we are set!
When deploying an ARM-tempalate, the Azure Virtual Network got updated with the new DNS server. The Virtual machines deployed before this update, still got the old DNS client settings, before the VM gets rebooted. This becomes a bit of a challange, when trying to domain join the computer with Desired State Configuration (DSC), since it will not find the domain controller (if not the old DNS settings where already pointed to a current domain controller).
To force a reboot thru DSC can be achieved by the module xPendingReboot and DSC Script. xPendingReboot checks if there is a pending reboot of the machine and works in conjunction with the Local Configuration Manager (LCM). If LCM is set to RebootNodeIfNeeded=$true, it will reboot. However, a new machine got no pending reboot. To achieve this we will need to set DSCMachineStatus to 1 and write a small registry item. This will indicate there is a PendingReboot.
First we need to load the module (including the domainJoin as well)
Import-DscResource -ModuleName xPendingReboot, xDSCDomainjoinSet the LCM RebootNodeIfNeeded to true
LocalConfigurationManager
{
RebootNodeIfNeeded = $true
}Create a xPendingReboot resource
xPendingReboot Reboot
{
Name = "Reboot"
}Add our “fake” reboot.
Script Reboot
{
TestScript = {
return (Test-Path HKLM:\SOFTWARE\MyMainKey\RebootKey)
}
SetScript = {
New-Item -Path HKLM:\SOFTWARE\MyMainKey\RebootKey -Force
$global:DSCMachineStatus = 1
}
GetScript = { return @{result = 'result'}}
}and finally the domainjoin with a DependsOn so it will happend before domainjoin.
xDSCDomainjoin JoinDomain
{
Domain = $DomainName
Credential = $DomainCreds
DependsOn = "[Script]Reboot"
}I stumpled on a problem that the Azure backups were not going thru correctly and this was because the Azure Recovery services could not get proper contact with the Azure agent on the virtual machine. The Virtual machine in this case is a Oracle Linux deployed from the Azure gallery.
Investigate
SSH to the machine and make sure you got an account with sudo permissions.
First of, checked if the waagent was running
sudo systemctl status waagentIn my case, it was not running. Next step was to consult the logs that is located in /var/log/waagent.log. For now, this was empty and the old logs were compress .gz files. To unpack the compressed files
gunzip waagent.log-20180201.gzAfter some searching around. The general tip was to update to latest version. To check current version, waagent has a switch for it. I also include the yum version that is the on the offical MS documentation sie
waagent -version
sudo yum list WALinuxAgentMy version was a low 2.0 version. So to check for updates, issue another yum command.
sudo yum check-update waagentThis will tell you if there is updates to your package.
Updating
To run the update, you issue yet another yum command
sudo yum install WALinuxAgentThis will update your waagent to latest version. This does not require a reboot.
Verifying
After we are done, we need to verify that the agent is running
sudo systemctl status waagentIn my case, it was running, but I got the following message
Warning: waagent.service changed on disk. Run 'systemctl daemon-reload' to reload units.To quote a excellent stackexchange answer,
taking changed configurations from filesystem and regenerating dependency trees
What does “systemctl daemon-reload” do?
So proceeding, we need to wrap up and issue the daemon reload command to reload defination files.
sudo systemctl daemon-reloadRunning the
sudo systemctl status waagentwill now not generate any errors – and the backups are running yet again.
Microsoft announced that ExpressRoute monitor is now GA! This is great news for all enterprises utilizing ExpressRoute, since there was no easy way to do this before.
ExpressRoute monitoring with Network Performance Monitor (NPM) is now generally available
The ExpressRoute monitor is based in Network Performance monitor (a solution in OMS) that Microsoft adds more and more value to. I thought I would dwelve a bit on the node part of the ExpressRoute monitoring.
The internals of the monitoring works with nodes on selected networks on-premise and in Azure sending packages to one another over port 8084 TCP (or port of your own choosing). Each hop on the way between your on-premise location and Azure are noted as one hop and nicely illustrated on your topology map
To get working nodes, you will need to install OMS agent on each node you want to have in your monitoring. I would recommend at least two at each network location, so a single node failure will not generate false-positives in your monitoring, however you will be fine with one.
Example setup
After the installation and registration, it usually takes a few minutes before popping up in your NPM. Under NPM –> Configuration, your nodes are located. Here you can add a location comment and check “Use for monitoring” for all the nodes you want to utilize in your ExpressRoute monitoring.
Note: If the OMS agent are not able to traverse with TCP 8084, only ICMP will been seen under Supported Protocol.
After adding nodes, go to Networks under configuration. By default each node will add it own subnet and get placed in default. So for example if Berlin node is in 192.168.10.0/24, that network will be added in default – hence to make it clearar we can set label on the networks. We can here choose a “Add Network”, and give a better network name, for example Berlin. Repeat for each network you have. Do not forget to press Save (happend to a friend…)
When done, we go back to NPM and choose Topology Dashboard and when we choose our nonde, we will get the network name
We can further divide our network into subnetworks, in case you have several networks on the same location, to make it even more granular.
With Azure ARM templates we have the possibility to add conditions. Meaning that we can for example decide if we want to deploy a certain resource or not that is defined in the JSON-template. This gives us great flexibility when authoring ARM templates.
An example is that you might decide to have an option to create a VPN-Gateway for your Network template and want to option to say Yes or No as input. If in the deployment you choose No, you do not really want to enter all the other parameters required for VPN, even if they are defined in your ARM Template. The parameter file you can choose as input (and is created by default if you work with Visual Studio) handles this null values. So far so good.
However, sometimes, we want to solve this from Powershell and inject parameters directly, without needing a parameter file. To accomplish this, we will need to create a hash table with the relevant parameters. For example we have two parameters. VPN, that is yes or no. If we choose no, we do not really want enter a VPNSharedKey, since it will be not needed.
First we enter the input parameters to the PowerShell script. The script gives the operator the possibility to choose two answers on VPN, yes or No.
[Cmdletbinding()]
Param
(
[Parameter(Mandatory=$true)]
[ValidateSet('yes','no')]
[string[]]$VPN,
[Parameter(Mandatory=$false)]
[String]$VPNSharedKey
)Then we pass this variables into our hash table
$Parameters @{
VPN ="$VPN"
VPNSharedKey ="$VPNSharedKey"
}Hence we can launch the Powershell script (Azuredeploy.json being our template file)
.\Deploy-ToAzure.ps1 -VPN no -TemplateFile C:\Azuredeploy.jsonFor further reading, see Microsoft documentation about splatting about_Splatting