Platform For AI:Optimization report

After you call the blade.optimize method to optimize a model, an optimization report is generated and contains parameters in the JSON format. The following sample code shows the structure of the optimization report:

Report: {
  // The software environment information, including the framework and Compute Unified Device Architecture (CUDA). 
  "software_context": [
    {
      "software": "tensorflow",
      "version": "1.15.0"
    },
    {
      "software": "cuda",
      "version": "9.0.176"
    }
  ],
  // The hardware environment information. 
  "hardware_context": {
    "device_type": "gpu",
    "microarchitecture": "T4"
  },
  "user_config": "",
  // The diagnostic information. 
  "diagnosis": {
    "model": "tmp_graph.pbtxt",           
    "test_data_source": "user provided",  
    "shape_variation": "dynamic",         
    "message": "",
    "test_data_info": "input_ids_a_1:0 shape: (1, 9240) data type: int32"
  },
  // The optimization items that achieve optimization effects. 
  "optimizations": [
    {
      "name": "TfStripUnusedNodes", 
      "status": "effective",        
      "speedup": "na",              
      "pre_run": "na",              
      "post_run": "na"              
    },
    {
      "name": "TfAutoMixedPrecisionGpu",
      "status": "effective",
      "speedup": "1.42",
      "pre_run": "9.35 ms",
      "post_run": "6.59 ms"
    }
  ],
  // The end-to-end optimization results. 
  "overall": {
    "baseline": "10.00 ms",   
    "optimized": "4.38 ms",   
    "speedup": "2.28"         
  },
  // The model information. 
  "model_info": {
    "input_format": "frozen_pb"  
  },
  
  // The compatibility information. 
  "compatibility_list": [
    {
      "device_type": "gpu",
      "microarchitecture": "T4"
    }
  ],
  "model_sdk": {}
}

The following table describes the parameters in the optimization report.

Parameter		Description
software_context		The software environment information, including the framework and CUDA.
hardware_context		The hardware environment information, including the device type and specifications.
user_config		The user configuration information. In an optimization report generated by PAI-Blade, this parameter is left empty.
diagnosis	model	The name of the model file.
	test_data_source	The source of the test data. Valid values: user provided: The test data is provided by the user. blade deduced: The test data is inferred by PAI-Blade.
	shape_variation	The variation of input shapes. Valid values: static multiple_static_shapes dynamic
	message	The error message returned if PAI-Blade fails to infer test data.
	test_data_info	The information about the test data, including the shape and data type.
optimizations	name	The name of the optimization item.
	status	Indicates whether the optimization takes effect. Valid values: effective: The optimization takes effect. ineffective: The optimization does not take effect.
	speedup	The acceleration ratio, which is calculated by using the following formula: speedup = pre_run/post_run.
	pre_run	The speed before optimization.
	post_run	The speed after optimization.
overall	baseline	The latency before acceleration.
	optimized	The latency after acceleration.
	speedup	The acceleration ratio, which is calculated by using the following formula: speedup = baseline/optimized.
model_info	input_format	The format of the original model.
compatibility_list		The compatibility information. To ensure that the optimized model can be properly run and deliver the expected performance, you must run the model on the device described in the compatibility information.
model_sdk		The additional SDK information required for model deployment. This parameter is left empty.