SAP HANA Scale-out Deployment Guide
1. Supported the installation of ECS Metrics Collector.
2. Adjusted the layout of some pages.
Optimized Alibaba Cloud Network Attached Storage (NAS) parameters.
This topic describes SAP HANA scale-out deployment on Alibaba Cloud. The deployment architecture is based on non-shared storage with one master node and two worker nodes. By using the automatic recovery feature of Elastic Compute Service (ECS), this architecture supports automatic recovery of failed nodes. Therefore, you do not need secondary nodes for SAP HANA instances.
China (Beijing) Zone F
SAP and hosts
SAP HANA (SID: BWS)
Users and groups
Ensure that all nodes in the SAP HANA cluster use the same user ID and user group ID.
User ID: Set the
[sid]adm user ID to 2000.
Group ID: Set the sapsys group ID to 2000.
The swap space is required for installing SAP applications and SAP HANA. We recommend that you create an SSD to provide the swap space when you create an ECS instance. For more information about swap space, see SAP Note 1597355 - Swap-space recommendation for Linux. An SAP service account is required to access this page.
Physical memory (RAM)
Recommended swap space
< 32 GB
2 times the amount of RAM
32 to 63 GB
64 to 127 GB
128 to 255 GB
256 to 511 GB
512 to 1,023 GB
1,024 to 2,047 GB
2,048 to 4,095 GB
4,096 to 8,191 GB
> 8,192 GB
We recommend that you use Autofs to mount global file systems. For more information about file system sizing for SAP HANA, see SAP HANA TDI-Storage Requirements.In this example, the size of the ECS physical memory (RAM) is 128 GB. The following file system sizing is only for reference:
Sizedata = 3 x RAM
[systems ≤ 512 GB] Sizeredolog = 1/2 x RAM
[systems > 512 GB] Sizeredolog(min) = 512 GB
Sizeinstallation(scale-out) = 1 x RAM_of_worker per 4 worker nodes
Sizebackups ≥ Sizedata + Sizeredolog
Alibaba Cloud account
If you do not have an Alibaba Cloud account, register one on the Alibaba Cloud official website or in the Alibaba Cloud app. You need to use a mobile number to register the account and complete real-name verification for the account. Then, you can log on to the Alibaba Cloud app with this account to manage and monitor your cloud resources, perform authentication, and ask questions and acquire knowledge in Yunqi Community.
Virtual Private Cloud (VPC) is an isolated network environment built on Alibaba Cloud. VPCs are logically isolated from one another. A VPC is a private network dedicated to you on Alibaba Cloud. You can configure the IP address range, routing table, and gateway to customize your VPC. For more information, see Virtual Private Cloud.
The following figure shows how to create a VPC as planned.
The following figure shows how to create a VSwitch and specify a CIDR block for the business network as planned.
ECS is a basic cloud computing service provided by Alibaba Cloud.You can log on to the ECS console or Alibaba Cloud app to configure your ECS instances. For more information about SAP NetWeaver and SAP HANA on Alibaba Cloud, see Note 1380654 - SAP support in cloud environments.
1. Create an ECS instance.
Create an ECS instance in the ECS console. Specify the billing method and zone.
Select an existing VPC and an existing security group for the ECS instance. For more information about the security group, see Security group FAQ.
Allows ICMP requests.
Allows remote Linux SSH access.
Allows requests from default Telnet ports.
Allows access from HTTPS websites.
Allows access from SAP HANA.
Allows access from VNC Viewer.
Specify the number and capacity of disks to be created as planned. For more information about disks, see Block storage performance.
We recommend that you use an ultra disk as the system disk and SSDs as data disks, and create an SSD to provide the swap space.
Specify the image.
Select the SUSE Linux Enterprise Server for SAP Applications 12 SP2 image from the image marketplace.
(Optional) Specify the RAM users as required. For more information about RAM, see RAM introduction
Check all configuration items and ensure that they are correct. Then, create the ECS instance. According to the planning of this deployment, you need to create three ECS instances. After the three ECS instances are created, update hostnames or private IP addresses as planned.
ECS Metrics Collector
ECS Metrics Collector is a monitor agent that the SAP system uses on Alibaba Cloud to collect required information about virtual machine configuration and underlying physical resource usage.
When the SAP system runs in an ECS instance, the SAP Host Agent uses the metadata service and APIs to obtain the information required for monitoring the SAP system, including the information about the operating system, network, storage, and SAP architecture. Then, the SAP Host Agent provides the information for SAP applications to analyze events and system performance.
You need to install ECS Metrics Collector for SAP for each ECS instance in which the SAP system is running, either for SAP applications or SAP HANA.
For more information, see ECS Metrics Collector for SAP deployment guide.
Alibaba Cloud NAS is a file storage service for compute nodes, such as ECS instances, Elastic High Performance Computing (E-HPC) clusters, and Container Service clusters. NAS complies with standard file access protocols. Without modifying existing applications, you can have a distributed file system that features unlimited capacity and performance scaling, a single namespace, shared access, high reliability, and high availability. For more information about NAS, see Network Attached Storage.
1. Create an NAS file system.
Select a region and a storage type. This deployment uses the NAS Performance storage type. For more information about NAS performance, see Storage types.
In the ECS console, choose Storage & Snapshots > NAS. On the page that appears, specify the region, storage type, capacity, and zone. Select Create new FS and bind naspackage. The following figure shows the configuration for this example.
After you paid for the NAS storage package, go to the ECS console and refresh the console. The created file system is displayed on the page.
2. Record the mount addresses of NAS file systems.
Create two NAS file systems for /hana/shared and /hana/backup as planned.
The following table lists the two NAS file systems in this example.
NAS file system
ECS automatic recovery
The automatic recovery feature is used to improve the availability of ECS instances. This feature reduces influences of physical failures on services.
In the event that the underlying hardware on which your ECS instances are hosted works abnormally or crashes unexpectedly, Alibaba Cloud sends an SMS message to you if the breakdown is confirmed as irreversible.
After the migration is complete, Alibaba Cloud will send a message to indicate that migration is completed. The ECS instance metadata, such as the instance ID, private network IP address, and public network IP address, will remain unchanged.
SAP HANA installation
Modify hostnames for all nodes of SAP applications and SAP HANA.
Add the following information to the hosts file in the /etc directory as planned:
192.168.10.4 hana01 192.168.10.5 hana02 192.168.10.6 hana03
Create file systems
Create NAS file systems for /hana/shared and /hana/backup. Create local XFS file systems for /hana/data and /hana/log.
1. Create the /hana/data and /hana/log file systems.
(1) Check disks.
(2) Create a physical volume.
(3) Create the hanavg volume group.
(4) Create the datalv and loglv logical volumes.
(5) Create the /hana/data and /hana/log file systems.
(6) Add mount points for the file systems. Enable the file systems to be mounted upon system startup.
fdisk -l pvcreate /dev/vdb /dev/vdc /dev/vdd vgcreate hanavg /dev/vdb /dev/vdc /dev/vdd lvcreate -L 384G -n datalv -i 3 -I 64 hanavg lvcreate -L 64G -n loglv -i 3 -I 64 hanavg mkfs.xfs /dev/hanavg/datalv mkfs.xfs /dev/hanavg/loglv mkdir -p /hana/data mkdir -p /hana/log
Run the vi /etc/fstab command to edit the fstab file as follows:
/dev/mapper/hanavg-datalv /hana/data xfs nobarrier,noatime,nodiratime,logbsize=256k 0 0 /dev/mapper/hanavg-loglv /hana/log xfs nobarrier,noatime,nodiratime,logbsize=256k 0 0
Run the mount -a command to mount all file systems.
2. Create the swap space.
(1) Check the disk used to provide the swap space.
Run the lsblk command to obtain disk information. In this example, /dev/vdd provides the swap space.
(2) Configure the swap space as follows:
mkswap /dev/vde swapon /dev/vde swapon -s # Checks the size of the swap space.#
3. Mount global file systems.
We recommend that you use Autofs to mount the /hana/shared and /hana/backup file systems. In this way, you do not need to create directories as mount points.
To configure Autofs, follow these steps:
(1) Run the following command to edit the auto.master file:
Add /- /etc/auto.nfs.
(2) Create and edit the auto.nfs file in the /etc directory as follows:
/hana/shared -rw,hard,intr,noresvport,timeo=60,retrans=2 194874b95d-qgi86.cn-beijing.nas.aliyuncs.com:/ /hana/backup -rw,hard,intr,noresvport,timeo=60,retrans=2 1207e4a953-gus54.cn-beijing.nas.aliyuncs.com:/
(3) Run the following command to start Autofs:
#systemctl start autofs
(4) Run the following command to enable Autofs to mount file systems upon system startup:
#systemctl enable autofs
You can run the cd command to access the two file systems to check whether they are mounted.
Prepare the operating system and installation package
You need to configure all nodes of the cluster. The following procedure shows how to configure a node.
1. Install the package required by HA configuration and optimization.
Install the saptune package.
#zypper install -y saptune
Run the following command to check whether this package is installed:
zypper se saptune
2. Check the Network Time Protocol (NTP) service.
By default, the NTP service is enabled for Alibaba ECS instances. If the time zone of your ECS instances is not Asia/Shanghai, change the time zone and configure the NTP service. Ensure that all ECS instances enable the NTP service and use the same time zone.
3. Install saptune.
As an upgraded version of the sapconf tool, saptune is available to SUSE Linux Enterprise Server 12 SP2 and later versions. You can use saptune to tune parameters for operating systems and databases. This ensures better performance for SAP NetWeaver or SAP HANA.
The syntax is as follows:
Tune system according to SAP and SUSE notes: saptune note [ list | verify ] saptune note [ apply | simulate | verify | customise | revert ] NoteID
Tune system for all notes applicable to your SAP solution: saptune solution [ list | verify ] saptune solution [ apply | simulate | verify | revert ] SolutionName
For this deployment, SAP HANA is installed. The following figure shows the installation results of saptune.
Run the following command to start a daemon:
#saptune daemon start
For more information about saptune, see Prepare your Linux for your SAP solution with saptune or the official documentation for SUSE Linux Enterprise Server.
Install SAP HANA
Install HANA on the primary node.
hana01:/sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64 # ./hdblcm SAP HANA Lifecycle Management - SAP HANA Database 2.00.020.00.1500920972 ************************************************************************ Scanning software locations... Detected components: SAP HANA Database (2.00.020.00.1500920972) in /sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64/server SAP HANA AFL (incl.PAL,BFL,OFL,HIE) (2.00.020.0000.1500932993) in /sapcd/HANA2.0_SPS02_THIN/HDB_AFL_LINUX_X86_64/packages SAP HANA EML AFL (2.00.020.0000.1500932993) in /sapcd/HANA2.0_SPS02_THIN/HDB_EML_AFL_10_LINUX_X86_64/packages SAP HANA Database Client (220.127.116.119440855) in /sapcd/HANA2.0_SPS02_THIN/HDB_CLIENT_LINUX_X86_64/client Choose an action Index | Action | Description ----------------------------------------------- 1 | install | Install new system 2 | extract_components | Extract components 3 | Exit (do nothing) | Enter selected action index : 1 SAP HANA Database version '2.00.020.00.1500920972' will be installed. Select additional components for installation: Index | Components | Description ------------------------------------------------------------------------------------------------- 1 | all | All components 2 | server | No additional components 3 | client | Install SAP HANA Database Client version 18.104.22.1689440855 4 | afl | Install SAP HANA AFL (incl.PAL,BFL,OFL,HIE) version 2.00.020.0000.1500932993 5 | eml | Install SAP HANA EML AFL version 2.00.020.0000.1500932993 Enter comma-separated list of the selected indices : 2,3 Enter Installation Path [/hana/shared]: /hana/shared Enter Local Host Name [hana01]: Do you want to add hosts to the system? (y/n) [n]: Enter SAP HANA System ID: BWS Enter Instance Number : 00 Enter Local Host Worker Group [default]: Index | System Usage | Description ------------------------------------------------------------------------------- 1 | production | System is used in a production environment 2 | test | System is used for testing, not production 3 | development | System is used for development, not production 4 | custom | System usage is neither production, test nor development Select System Usage / Enter Index : 4 Enter Location of Data Volumes [/hana/data/BWS]: Enter Location of Log Volumes [/hana/log/BWS]: Restrict maximum memory allocation? [n]: Enter Certificate Host Name For Host 'hana01' [hana01]: Enter SAP Host Agent User (sapadm) Password: Confirm SAP Host Agent User (sapadm) Password: Enter System Administrator (bwsadm) Password: Confirm System Administrator (bwsadm) Password: Enter System Administrator Home Directory [/usr/sap/BWS/home]: Enter System Administrator Login Shell [/bin/sh]: Enter System Administrator User ID : 2000 Enter ID of User Group (sapsys) : 2000 Enter System Database User (SYSTEM) Password: Confirm System Database User (SYSTEM) Password: Restart system after machine reboot? [n]: Summary before execution: ========================= SAP HANA Database System Installation Installation Parameters Remote Execution: ssh Database Isolation: low Installation Path: /hana/shared Local Host Name: hana01 SAP HANA System ID: BWS Instance Number: 00 Local Host Worker Group: default System Usage: custom Location of Data Volumes: /hana/data/BWS Location of Log Volumes: /hana/log/BWS Certificate Host Names: hana01 -> hana01 System Administrator Home Directory: /usr/sap/BWS/home System Administrator Login Shell: /bin/sh System Administrator User ID: 2000 ID of User Group (sapsys): 2000 SAP HANA Database Client Installation Path: /hana/shared/BWS/hdbclient Software Components SAP HANA Database Install version 2.00.020.00.1500920972 Location: /sapcd/HANA2.0_SPS02_THIN/HDB_SERVER_LINUX_X86_64/server SAP HANA AFL (incl.PAL,BFL,OFL,HIE) Do not install SAP HANA EML AFL Do not install SAP HANA Database Client Install version 22.214.171.1249440855 Location: /sapcd/HANA2.0_SPS02_THIN/HDB_CLIENT_LINUX_X86_64/client Do you want to continue? (y/n):
Modify configurations to meet requirements of the non-shared storage architecture.
Add the following configuration:
Change the internal communication mode. Run the./hdblcm -action=configure_internal_network commandas a root user to specify the CIDR block for communication. In this example, the communication CIDR block is 192.168.10/24.
Install HANA on worker nodes. The following use one worker node as example. The installation of the other worker node is similar.
Create paths for instances.
mkdir -p /hana/data/
mkdir -p /hana/log/
Modify directory permissions.
chmod -R 777 /hana/data/
chmod -R 777 /hana/log/
Install SAP HANA on the worker node.
#cd /hana/shared/BWS/hdblcm #./hdblcm SAP HANA Lifecycle Management - SAP HANA Database 126.96.36.199 ********************************************************** Choose an action Index | Action | Description ----------------------------------------------------------------------------- 1 | add_hosts | Add Hosts to the SAP HANA Database System 2 | register_rename_system | Register and Rename SAP HANA Database System 3 | exit | Exit (do nothing) Enter selected action index : 1 Enter comma-separated host names to add: hana02 Enter SAP Host Agent User (sapadm) Password: Select roles for host 'hana02': Index | Host Role | Description ------------------------------------------------------------------- 1 | worker | Database Worker 2 | standby | Database Standby 3 | extended_storage_worker | Dynamic Tiering Worker 4 | extended_storage_standby | Dynamic Tiering Standby 5 | streaming | Streaming Analytics 6 | rdsync | Remote Data Sync 7 | ets_worker | Accelerator for SAP ASE Worker 8 | ets_standby | Accelerator for SAP ASE Standby 9 | xs_worker | XS Advanced Runtime Worker 10 | xs_standby | XS Advanced Runtime Standby Enter comma-separated list of selected indices : 1 Enter Host Failover Group for host 'hana02' [default]: Enter Storage Partition Number for host 'hana02' [<<assign automatically>>]: Enter Worker Group for host 'hana02' [default]: Enter System Administrator (bwsadm) Password: Enter Certificate Host Name For Host 'hana02' [hana02]: Summary before execution: ========================= Add Hosts to SAP HANA Database System Add Hosts Parameters Skip all SAP Host Agent calls: No Remote Execution: ssh Enable the installation or upgrade of the SAP Host Agent: Yes Auto Initialize Services: Yes Do not start added hosts and do not start SAP HANA Database System: No Certificate Host Names: hana02 -> hana02 Do not Modify '/etc/sudoers' File: No Additional Hosts hana02 Role: Database Worker (worker) High-Availability Group: default Worker Group: default Storage Partition: <<assign automatically>> Do you want to continue? (y/n): y
Modify parameters to enable SAP HANA to start upon system startup.
Log on to the SAP HANA hosts as a root user and modify the Autostart parameter.
vim /hana/shared/[SID]/profile/[SID]_HDB[instance number]_[host name]
Change the value of the Autostart parameter from 0 to 1. Save the settings and restart SAP HANA instances.
In this example, you need to modify the parameter for three instances.
Repeat the preceding steps to install SAP HANA on the other worker node according to the planning for this example.
Verify the installation
Log out the root account and log on to the ECS instances as the administrator.su -
Verify that the cluster is deployed.
watch python landscapeHostConfiguration.py