To create an HDInsight cluster by using Azure Power Shell, you must complete the following procedures:
Create an Azure resource group
Create an Azure Storage account
Create an Azure Blob container
Create an HDInsight cluster
The two most important parameters that you must set to create Linux clusters are the ones that specify the OS type and the SSH user details:
Make sure you specify the -OSType parameter as Linux.
To use SSH for remote sessions on the clusters, you can specify the SSH user password or the SSH public key. If you specify both the SSH user password and the SSH public key, the key will be ignored. If you want to use the SSH key for remote sessions, you must specify a blank SSH password when prompted for one. For information,
script begins
----------------------------------------------------------------------------------------
$token ="hadoopcluster"
$subscriptionID = "<SubscriptionName>" # Provide your Subscription Name
$resourceGroupName = "resource group name" # Provide a Resource Group name
$clusterName = $token
$defaultStorageAccountName = $token + "Storage account" # Provide a Storage account name
$defaultStorageContainerName = $token + "container"
$location = "East US 2" # Change the location if needed
$clusterNodes = 1 # The number of nodes in the HDInsight cluster
$sshCredentials = # provide your credentials for ssh
# Sign in to Azure
Login-AzureRmAccount
# Select the subscription to use if you have multiple subscriptions
Select-AzureRmSubscription -SubscriptionId $subscriptionID
# Create an Azure Resource Group
New-AzureRmResourceGroup -Name $resourceGroupName -Location $location
# Create an Azure Storage account and container used as the default storage
New-AzureRmStorageAccount `
-ResourceGroupName $resourceGroupName `
-StorageAccountName $defaultStorageAccountName `
-Location $location `
-Type Standard_LRS
$defaultStorageAccountKey = (Get-AzureRmStorageAccountKey -Name $defaultStorageAccountName -ResourceGroupName $resourceGroupName)[0].Value
$destContext = New-AzureStorageContext -StorageAccountName $defaultStorageAccountName -StorageAccountKey $defaultStorageAccountKey
New-AzureStorageContainer -Name $defaultStorageContainerName -Context $destContext
# Create an HDInsight cluster
$credentials = Get-Credential -Message "Enter Cluster user credentials" -UserName "admin"
$sshCredentials = Get-Credential -Message "Enter SSH user credentials"
# The location of the HDInsight cluster must be in the same data center as the Storage account.
$location = Get-AzureRmStorageAccount -ResourceGroupName $resourceGroupName -StorageAccountName $defaultStorageAccountName | %{$_.Location}
New-AzureRmHDInsightCluster `
-ClusterName $clusterName `
-ResourceGroupName $resourceGroupName `
-HttpCredential $credentials `
-Location $location `
-DefaultStorageAccountName "$defaultStorageAccountName.blob.core.windows.net" `
-DefaultStorageAccountKey $defaultStorageAccountKey `
-DefaultStorageContainer $defaultStorageContainerName `
-ClusterSizeInNodes $clusterNodes `
-ClusterType Hadoop `
-OSType Linux `
-Version "3.4" `
-SshCredential $sshCredentials
write-host "installation completed"
