You can use the LLM-LaTeX Remove Comments (DLC) component to process TeX text data. The component removes the comments in LaTeX text. The input Object Storage Service (OSS) data file must be in the JSON Lines format. Each line in the file is a valid JSON object, but the file as a whole is not a valid JSON object. You can click here to view an example.
Supported computing resources
Algorithm
This component removes strings that match specific regular expressions. The following table describes the regular expressions.
Comment type | Regular expression |
Comment lines |
|
Inline comments |
|
This component extracts all strings that match the preceding regular expression and replaces the strings with an empty string. Example:
Before processing
| After processing
|
Configure the component
Configure the parameters of the LLM-LaTeX Remove Comments (DLC) component on the pipeline page of Machine Learning Designer in the Platform for AI (PAI) console. The following table describes the parameters.
Tab | Parameter | Required | Description | Default value | |
Fields Setting | Target Process Field | Yes | The name of the field that you want to process. | No default value | |
Whether remove all line comments | No | Specifies whether to remove all comment lines. | Selected | ||
Whether remove all in comments within a line | No | Specifies whether to remove all in comments within a line. | Selected | ||
OSS Directory for Saving OutputData | No | The OSS directory in which the generated data is stored. If you do not specify this parameter, the default path of the workspace is used. | No default value | ||
Tuning | Number of Processes | No | The number of processes. | 8 | |
Select Resource Group | Public Resource Group | No | The instance type (CPU or GPU), number of instances, and a virtual private cloud (VPC) that you want to use. | No default value | |
Dedicated resource group | No | The number of vCPUs, memory, shared memory, number of GPUs, and number of instances that you want to use. | No default value | ||
Maximum Running Duration (seconds) | No | The maximum period of time the component can run. If this period of time is exceeded, the job is terminated. | No default value | ||