SAP:SAP HANA Scale-out Deployment Guide

SAP HANA Scale-out Deployment Guide

• Overview

• Architecture

• Resource planning

• Preparations

  • Alibaba Cloud account

  • VPC

  • ECS instance

  • ECS Metrics Collector

  • NAS

  • ECS automatic recovery

• SAP HANA installation

  • Modify hostnames

  • Create file systems

  • Prepare the operating system and installation package

  • Install SAP HANA

• Verify the installation

Release history

Overview

This topic describes SAP HANA scale-out deployment on Alibaba Cloud. The deployment architecture is based on non-shared storage with one master node and two worker nodes. By using the automatic recovery feature of Elastic Compute Service (ECS), this architecture supports automatic recovery of failed nodes. Therefore, you do not need secondary nodes for SAP HANA instances.

Architecture

Resource planning

Network

SAP and hosts

SAP HANA (SID: BWS)

Users and groups

Ensure that all nodes in the SAP HANA cluster use the same user ID and user group ID.

User ID: Set the [sid]adm user ID to 2000.

Group ID: Set the sapsys group ID to 2000.

Swap space

The swap space is required for installing SAP applications and SAP HANA. We recommend that you create an SSD to provide the swap space when you create an ECS instance. For more information about swap space, see SAP Note 1597355 - Swap-space recommendation for Linux. An SAP service account is required to access this page.

File systems

We recommend that you use Autofs to mount global file systems. For more information about file system sizing for SAP HANA, please refer to the SAP official documentation. In this example, the size of the ECS physical memory (RAM) is 128 GB. The following file system sizing is only for reference:

Sizedata = 3 x RAM

[systems ≤ 512 GB] Sizeredolog = 1/2 x RAM

[systems > 512 GB] Sizeredolog(min) = 512 GB

Sizeinstallation(scale-out) = 1 x RAM_of_worker per 4 worker nodes

Sizebackups ≥ Sizedata + Sizeredolog

Preparations

Alibaba Cloud account

If you do not have an Alibaba Cloud account, register one on the Alibaba Cloud official website or in the Alibaba Cloud app. You need to use a mobile number to register the account and complete real-name verification for the account. Then, you can log on to the Alibaba Cloud app with this account to manage and monitor your cloud resources, perform authentication, and ask questions and acquire knowledge in Yunqi Community.

VPC

Virtual Private Cloud (VPC) is an isolated network environment built on Alibaba Cloud. VPCs are logically isolated from one another. A VPC is a private network dedicated to you on Alibaba Cloud. You can configure the IP address range, routing table, and gateway to customize your VPC. For more information, see Virtual Private Cloud.

The following figure shows how to create a VPC as planned.

The following figure shows how to create a vSwitch and specify a CIDR block for the business network as planned.

ECS instance

ECS is a basic cloud computing service provided by Alibaba Cloud.You can log on to the ECS console or Alibaba Cloud app to configure your ECS instances. For more information about SAP NetWeaver and SAP HANA on Alibaba Cloud, see Note 1380654 - SAP support in cloud environments.

1. Create an ECS instance.

Create an ECS instance in the ECS console. Specify the billing method and zone.

Select an existing VPC and an existing security group for the ECS instance. For more information about the security group, see Security group FAQ.

The following table describes the security group rules for this example.

Specify the number and capacity of disks to be created as planned. For more information about disks, see Block storage performance.

We recommend that you use an ultra disk as the system disk and SSDs as data disks, and create an SSD to provide the swap space.

1. Specify the image.

   Select the SUSE Linux Enterprise Server for SAP Applications 12 SP2 image from the image marketplace.

(Optional) Specify the RAM users as required.

Check all configuration items and ensure that they are correct. Then, create the ECS instance. According to the planning of this deployment, you need to create three ECS instances. After the three ECS instances are created, update hostnames or private IP addresses as planned.

ECS Metrics Collector

ECS Metrics Collector is a monitor agent that the SAP system uses on Alibaba Cloud to collect required information about virtual machine configuration and underlying physical resource usage.

When the SAP system runs in an ECS instance, the SAP Host Agent uses the metadata service and APIs to obtain the information required for monitoring the SAP system, including the information about the operating system, network, storage, and SAP architecture. Then, the SAP Host Agent provides the information for SAP applications to analyze events and system performance.

You need to install ECS Metrics Collector for SAP for each ECS instance in which the SAP system is running, either for SAP applications or SAP HANA.

For more information, see ECS Metrics Collector for SAP deployment guide.

NAS

Alibaba Cloud NAS is a file storage service for compute nodes, such as ECS instances, Elastic High Performance Computing (E-HPC) clusters, and Container Service clusters. NAS complies with standard file access protocols. Without modifying existing applications, you can have a distributed file system that features unlimited capacity and performance scaling, a single namespace, shared access, high reliability, and high availability. For more information about NAS, see Network Attached Storage.

1. Create an NAS file system.

Select a region and a storage type. This deployment uses the NAS Performance storage type. For more information about NAS performance, see Storage types.

In the ECS console, choose Storage & Snapshots > NAS. On the page that appears, specify the region, storage type, capacity, and zone. Select Create new FS and bind naspackage. The following figure shows the configuration for this example.

After you paid for the NAS storage package, go to the ECS console and refresh the console. The created file system is displayed on the page.

2. Record the mount addresses of NAS file systems.

Create two NAS file systems for /hana/shared and /hana/backup as planned.

The following table lists the two NAS file systems in this example.

ECS automatic recovery

The automatic recovery feature is used to improve the availability of ECS instances. This feature reduces influences of physical failures on services.

In the event that the underlying hardware on which your ECS instances are hosted works abnormally or crashes unexpectedly, Alibaba Cloud sends an SMS message to you if the breakdown is confirmed as irreversible.

After the migration is complete, Alibaba Cloud will send a message to indicate that migration is completed. The ECS instance metadata, such as the instance ID, private network IP address, and public network IP address, will remain unchanged.

For more information, see Automatic recovery of instances.

SAP HANA installation

Modify hostnames

Modify hostnames for all nodes of SAP applications and SAP HANA.

Add the following information to the hosts file in the /etc directory as planned:

192.168.10.4    hana01
192.168.10.5    hana02
192.168.10.6    hana03

Create file systems

Create NAS file systems for /hana/shared and /hana/backup. Create local XFS file systems for /hana/data and /hana/log.

1. Create the /hana/data and /hana/log file systems.

(1) Check disks.

(2) Create a physical volume.

(3) Create the hanavg volume group.

(4) Create the datalv and loglv logical volumes.

(5) Create the /hana/data and /hana/log file systems.

(6) Add mount points for the file systems. Enable the file systems to be mounted upon system startup.

fdisk -l
pvcreate /dev/vdb /dev/vdc /dev/vdd
vgcreate hanavg /dev/vdb /dev/vdc /dev/vdd
lvcreate -L 384G -n datalv -i 3 -I 64 hanavg
lvcreate -L 64G -n loglv -i 3 -I 64 hanavg

mkfs.xfs /dev/hanavg/datalv
mkfs.xfs /dev/hanavg/loglv
mkdir -p /hana/data
mkdir -p /hana/log

Run the vi /etc/fstab command to edit the fstab file as follows:

/dev/mapper/hanavg-datalv       /hana/data      xfs     nobarrier,noatime,nodiratime,logbsize=256k     0       0
/dev/mapper/hanavg-loglv        /hana/log       xfs     nobarrier,noatime,nodiratime,logbsize=256k     0       0

Run the mount -a command to mount all file systems.

2. Create the swap space.

(1) Check the disk used to provide the swap space.

Run the lsblk command to obtain disk information. In this example, /dev/vdd provides the swap space.

(2) Configure the swap space as follows:

mkswap  /dev/vde
swapon  /dev/vde
swapon -s # Checks the size of the swap space.#

3. Mount global file systems.

We recommend that you use Autofs to mount the /hana/shared and /hana/backup file systems. In this way, you do not need to create directories as mount points.

To configure Autofs, follow these steps:

(1) Run the following command to edit the auto.master file:

#vim /etc/auto.master

Add /- /etc/auto.nfs.

(2) Create and edit the auto.nfs file in the /etc directory as follows:

/hana/shared -rw,hard,intr,noresvport,timeo=60,retrans=2      194874b95d-qgi86.cn-beijing.nas.aliyuncs.com:/
/hana/backup -rw,hard,intr,noresvport,timeo=60,retrans=2     1207e4a953-gus54.cn-beijing.nas.aliyuncs.com:/

(3) Run the following command to start Autofs:

#systemctl start autofs

(4) Run the following command to enable Autofs to mount file systems upon system startup:

#systemctl enable autofs

You can run the cd command to access the two file systems to check whether they are mounted.

Prepare the operating system and installation package

You need to configure all nodes of the cluster. The following procedure shows how to configure a node.

1. Install the package required by HA configuration and optimization.

Install the saptune package.

#zypper install -y saptune

Run the following command to check whether this package is installed:

zypper se saptune

2. Check the Network Time Protocol (NTP) service.

#ntpq -p

By default, the NTP service is enabled for Alibaba ECS instances. If the time zone of your ECS instances is not Asia/Shanghai, change the time zone and configure the NTP service. Ensure that all ECS instances enable the NTP service and use the same time zone.

3. Install saptune.

As an upgraded version of the sapconf tool, saptune is available to SUSE Linux Enterprise Server 12 SP2 and later versions. You can use saptune to tune parameters for operating systems and databases. This ensures better performance for SAP NetWeaver or SAP HANA.

The syntax is as follows:

SAP note

Tune system according to SAP and SUSE notes:
  saptune note [ list | verify ]
  saptune note [ apply | simulate | verify | customise | revert ] NoteID

SAP solution

Tune system for all notes applicable to your SAP solution:
  saptune solution [ list | verify ]
  saptune solution [ apply | simulate | verify | revert ] SolutionName

For this deployment, SAP HANA is installed. The following figure shows the installation results of saptune.

Run the following command to start a daemon:

#saptune daemon start

For more information about saptune, see Prepare your Linux for your SAP solution with saptune or the official documentation for SUSE Linux Enterprise Server.

Install SAP HANA

1. Install HANA on the primary node.

hana01:/sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64 # ./hdblcm


SAP HANA Lifecycle Management - SAP HANA Database 2.00.020.00.1500920972
************************************************************************


Scanning software locations...
Detected components:
    SAP HANA Database (2.00.020.00.1500920972) in /sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64/server
    SAP HANA AFL (incl.PAL,BFL,OFL,HIE) (2.00.020.0000.1500932993) in /sapcd/HANA2.0_SPS02_THIN/HDB_AFL_LINUX_X86_64/packages
    SAP HANA EML AFL (2.00.020.0000.1500932993) in /sapcd/HANA2.0_SPS02_THIN/HDB_EML_AFL_10_LINUX_X86_64/packages
    SAP HANA Database Client (2.2.23.1499440855) in /sapcd/HANA2.0_SPS02_THIN/HDB_CLIENT_LINUX_X86_64/client



Choose an action

  Index | Action             | Description
  -----------------------------------------------
  1     | install            | Install new system
  2     | extract_components | Extract components
  3     | Exit (do nothing)  | 

Enter selected action index [3]: 1


SAP HANA Database version '2.00.020.00.1500920972' will be installed.

Select additional components for installation:

  Index | Components | Description
  -------------------------------------------------------------------------------------------------
  1     | all        | All components
  2     | server     | No additional components
  3     | client     | Install SAP HANA Database Client version 2.2.23.1499440855
  4     | afl        | Install SAP HANA AFL (incl.PAL,BFL,OFL,HIE) version 2.00.020.0000.1500932993
  5     | eml        | Install SAP HANA EML AFL version 2.00.020.0000.1500932993

Enter comma-separated list of the selected indices [3]: 2,3   
Enter Installation Path [/hana/shared]: /hana/shared
Enter Local Host Name [hana01]: 
Do you want to add hosts to the system? (y/n) [n]: 
Enter SAP HANA System ID: BWS
Enter Instance Number [00]: 00
Enter Local Host Worker Group [default]: 

  Index | System Usage | Description
  -------------------------------------------------------------------------------
  1     | production   | System is used in a production environment
  2     | test         | System is used for testing, not production
  3     | development  | System is used for development, not production
  4     | custom       | System usage is neither production, test nor development

Select System Usage / Enter Index [4]: 4
Enter Location of Data Volumes [/hana/data/BWS]: 
Enter Location of Log Volumes [/hana/log/BWS]: 
Restrict maximum memory allocation? [n]: 
Enter Certificate Host Name For Host 'hana01' [hana01]: 
Enter SAP Host Agent User (sapadm) Password: 
Confirm SAP Host Agent User (sapadm) Password: 
Enter System Administrator (bwsadm) Password: 
Confirm System Administrator (bwsadm) Password: 
Enter System Administrator Home Directory [/usr/sap/BWS/home]: 
Enter System Administrator Login Shell [/bin/sh]: 
Enter System Administrator User ID [1001]: 2000
Enter ID of User Group (sapsys) [79]: 2000
Enter System Database User (SYSTEM) Password: 
Confirm System Database User (SYSTEM) Password: 
Restart system after machine reboot? [n]: 

Summary before execution:
=========================

SAP HANA Database System Installation
   Installation Parameters
      Remote Execution: ssh
      Database Isolation: low
      Installation Path: /hana/shared
      Local Host Name: hana01
      SAP HANA System ID: BWS
      Instance Number: 00
      Local Host Worker Group: default
      System Usage: custom
      Location of Data Volumes: /hana/data/BWS
      Location of Log Volumes: /hana/log/BWS
      Certificate Host Names: hana01 -> hana01
      System Administrator Home Directory: /usr/sap/BWS/home
      System Administrator Login Shell: /bin/sh
      System Administrator User ID: 2000
      ID of User Group (sapsys): 2000
      SAP HANA Database Client Installation Path: /hana/shared/BWS/hdbclient
   Software Components
      SAP HANA Database
         Install version 2.00.020.00.1500920972
         Location: /sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64/server
      SAP HANA AFL (incl.PAL,BFL,OFL,HIE)
         Do not install
      SAP HANA EML AFL
         Do not install
      SAP HANA Database Client
         Install version 2.2.23.1499440855
         Location: /sapcd/HANA2.0_SPS02_THIN/HDB_CLIENT_LINUX_X86_64/client

Do you want to continue? (y/n):

Modify configurations to meet requirements of the non-shared storage architecture.

vim /hana/shared/[SID]/global/hdb/custom/config/global.ini

Add the following configuration:

basepath_shared=no

Change the internal communication mode. Run the./hdblcm -action=configure_internal_network commandas a root user to specify the CIDR block for communication. In this example, the communication CIDR block is 192.168.10/24.

1. Install HANA on worker nodes. The following use one worker node as example. The installation of the other worker node is similar.

   Create paths for instances.

   mkdir -p /hana/data/[HANA SID]

   mkdir -p /hana/log/[HANA SID]

   Modify directory permissions.

   chmod -R 777 /hana/data/[HANA SID]

   chmod -R 777 /hana/log/[HANA SID]

Install SAP HANA on the worker node.

Run the cd command as a root user to access the installation path.cd /hana/shared/[HANA SID]/hdblcm

#cd /hana/shared/BWS/hdblcm
#./hdblcm

SAP HANA Lifecycle Management - SAP HANA Database 2.2.18.0
**********************************************************

Choose an action

  Index | Action                 | Description
  -----------------------------------------------------------------------------
  1     | add_hosts              | Add Hosts to the SAP HANA Database System
  2     | register_rename_system | Register and Rename SAP HANA Database System
  3     | exit                   | Exit (do nothing)

Enter selected action index [3]: 1

Enter comma-separated host names to add: hana02
Enter SAP Host Agent User (sapadm) Password: 

Select roles for host 'hana02':

  Index | Host Role                | Description
  -------------------------------------------------------------------
  1     | worker                   | Database Worker 
  2     | standby                  | Database Standby 
  3     | extended_storage_worker  | Dynamic Tiering Worker 
  4     | extended_storage_standby | Dynamic Tiering Standby 
  5     | streaming                | Streaming Analytics 
  6     | rdsync                   | Remote Data Sync 
  7     | ets_worker               | Accelerator for SAP ASE Worker 
  8     | ets_standby              | Accelerator for SAP ASE Standby 
  9     | xs_worker                | XS Advanced Runtime Worker 
  10    | xs_standby               | XS Advanced Runtime Standby 

Enter comma-separated list of selected indices [1]: 1
Enter Host Failover Group for host 'hana02' [default]: 
Enter Storage Partition Number for host 'hana02' [<<assign automatically>>]: 
Enter Worker Group for host 'hana02' [default]: 
Enter System Administrator (bwsadm) Password: 
Enter Certificate Host Name For Host 'hana02' [hana02]: 

Summary before execution:
=========================

Add Hosts to SAP HANA Database System
   Add Hosts Parameters
      Skip all SAP Host Agent calls: No
      Remote Execution: ssh
      Enable the installation or upgrade of the SAP Host Agent: Yes
      Auto Initialize Services: Yes
      Do not start added hosts and do not start SAP HANA Database System: No
      Certificate Host Names: hana02 -> hana02
      Do not Modify '/etc/sudoers' File: No
   Additional Hosts
      hana02
         Role: Database Worker (worker)
         High-Availability Group: default
         Worker Group: default
         Storage Partition: <<assign automatically>>

Do you want to continue? (y/n): y

1. Modify parameters to enable SAP HANA to start upon system startup.

   Log on to the SAP HANA hosts as a root user and modify the Autostart parameter.

   vim /hana/shared/[SID]/profile/[SID]_HDB[instance number]_[host name]

   Change the value of the Autostart parameter from 0 to 1. Save the settings and restart SAP HANA instances.

   In this example, you need to modify the parameter for three instances.

   

Repeat the preceding steps to install SAP HANA on the other worker node according to the planning for this example.

Verify the installation

1. Log out the root account and log on to the ECS instances as the administrator.su -[sid]adm

2. Verify that the cluster is deployed.

   cd /hana/shared/[SID]/exe/linuxx86_64/hdb/python_support/

   watch python landscapeHostConfiguration.py