Log Service enables Logtail to collect logs from Kubernetes clusters and allows you to manage collection configurations by using CustomResourceDefinition (CRD). This topic describes how to install and use Logtail to collect logs from Kubernetes clusters.

Procedure

  1. Install the alibaba-log-controller Helm package.
  2. Create a collection configuration.

    You can create a collection configuration in the Log Service console or by using a CRD as required. This topic describes how to create a collection configuration in the console.

Figure 1. Procedure
Procedure

Step 1: Install the package

  1. Log on to the primary node of an Alibaba Cloud Container Service for Kubernetes cluster that you created.

    For more information, see Access Kubernetes clusters by using SSH key pairs.

  2. Replace parameters as required and run the installation command.
    Set ${your_k8s_cluster_id} to the ID of your Kubernetes cluster and run the following command:
    wget http://logtail-release-cn-hangzhou.oss-cn-hangzhou.aliyuncs.com/kubernetes/alicloud-log-k8s-install.sh -O alicloud-log-k8s-install.sh; chmod 744 ./alicloud-log-k8s-install.sh; sh ./alicloud-log-k8s-install.sh ${your_k8s_cluster_id}

Installation example

After the installation command is run, the following information is returned:

[root@iZbp******biaZ ~]# wget http://logtail-release-cn-hangzhou.oss-cn-hangzhou.aliyuncs.com/kubernetes/alicloud-log-k8s-install.sh -O alicloud-log-k8s-install.sh; chmod 744 ./alicloud-log-k8s-install.sh; sh ./alicloud-log-k8s-install.sh c12ba20**************86939f0b
...
...
...
alibaba-cloud-log/Chart.yaml
alibaba-cloud-log/templates/
alibaba-cloud-log/templates/_helpers.tpl
alibaba-cloud-log/templates/alicloud-log-crd.yaml
alibaba-cloud-log/templates/logtail-daemonset.yaml
alibaba-cloud-log/templates/NOTES.txt
alibaba-cloud-log/values.yaml
NAME: alibaba-log-controller
LAST DEPLOYED: Wed May 16 18:43:06 2018
NAMESPACE: default
STATUS: DEPLOYED


RESOURCES:
==> v1beta1/ClusterRoleBinding
NAME AGE
alibaba-log-controller 0s


==> v1beta1/DaemonSet
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
logtail 2 2 0 2 0  0s


==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
alibaba-log-controller 1 1 1 0 0s


==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
logtail-ff6rf 0/1 ContainerCreating 0 0s
logtail-q5s87 0/1 ContainerCreating 0 0s
alibaba-log-controller-7cf6d7dbb5-qvn6w 0/1 ContainerCreating 0 0s


==> v1/ServiceAccount
NAME SECRETS AGE
alibaba-log-controller 1 0s


==> v1beta1/CustomResourceDefinition
NAME AGE
aliyunlogconfigs.log.alibabacloud.com 0s


==> v1beta1/ClusterRole
alibaba-log-controller 0s




[SUCCESS] install helm package : alibaba-log-controller success.

You can run the helm status alibaba-log-controller command to check the status of pods in the Kubernetes cluster. If all pods are in the running state, Logtail is installed.

Then, Log Service creates a project whose name starts with k8s-log. You can use k8s-log as a keyword to search for this project in the Log Service console.

Step 2: Create a collection configuration

To create a Logstore and collect standard output logs from all containers in the Kubernetes cluster, follow these steps:
  1. Log on to the Log Service console and go to the Overview page.

    Click the project created in step 1 to go to the Overview page.

  2. Create a Logstore.
    In the left-side navigation pane, click the icon next to the search box. In the dialog box that appears, create a Logstore.
    Figure 2. Create a Logstore
    Create a Logstore
  3. Create a collection configuration.
    1. After creating the Logstore, click Data Import Wizard in the dialog box that appears.
    2. In the Import Data dialog box, select Docker Standard Output - Container.
    3. Create a server group.
      1. Install Logtail on Elastic Compute Service (ECS) instances.

        On the ECS Instances tab, select ECS instances and click Install. For more information about how to install Logtail in Windows, see Install Logtail in Windows.

      2. Create a server group.
        Before creating a server group, make sure that Logtail is installed.
        • Servers of Alibaba Group: By default, Logtail is installed on these servers. If Logtail is not installed on a server, contact Alibaba Cloud as prompted.
        • ECS instances: Select ECS instances and click Install. ECS instances that are running in Windows do not support one-click installation of Logtail. In this case, you need to manually install Logtail. For more information, see Install Logtail in Windows.
        • On-premises servers: Install Logtail as prompted. For more information about how to install Logtail, see Install Logtail in Linux or Install Logtail in Windows based on your operating system.
        After installing Logtail, click Complete Installation to create a server group. If you have created a server group, click Use Existing Server Groups.
    4. Configure the server group.

      Move the server group from Source Server Groups to Applied Server Groups.

    5. Configure the data source.

      In the Specify Data Source step, click Next. You can use the default configuration to collect standard output logs from all containers.

      Figure 3. Collect Docker standard output logs
      Collect Docker standard output logs
    6. Configure log query and analysis.

      By default, Full Text Index is enabled. You can also add key/value indexes or click Automatic Index Generation.

The collection configuration is created. To configure LogShipper, continue with the follow-up configuration as prompted.

View collected logs

Based on the created collection configuration, Logtail can collect standard output logs 1 minute after a container in your Kubernetes cluster writes data to the standard output stream. Return to the Overview page. Click the Overview icon next to the target Logstore name and select Consumption Preview to view the currently collected logs. You can also select Search & Analysis to query and analyze the collected logs.
Figure 4. Preview and query logs
Preview and query logs

To query and analyze logs on the Search & Analysis page, click a log field in the Quick Analysis section or enter a keyword in the search box, as shown in the following figure.

Figure 5. Query logs
Query logs

Other collection configuration methods

In addition to the preceding collection configuration created in the console for collecting standard output logs, Log Service also supports other collection configuration methods.

Create collection configurations in the console

You can also create the following collection configurations in the console:

Create collection configurations by using CRD

For more information, see Configure Kubernetes log collection on CRD.