Blanche
Engineer
Engineer
  • UID619
  • Fans2
  • Follows2
  • Posts59
Reads:1417Replies:0

Explore the ART World - dex2oat at a Glance

Created#
More Posted time:Sep 22, 2016 13:13 PM
dex2oat Procedure Analysis
Before learning about the procedure, let's take a look at the procedure diagram first to gain a general idea about the steps:


Main function
The main function of dex2oat is an encapsulation of the dex2oat factory function.
int main(int argc, char** argv) {
  int result = art::dex2oat(argc, argv);
  // Everything was done, do an explicit exit here to avoid running Runtime destructors that take
  // time (bug 10645725) unless we're a debug build or running on valgrind. Note: The Dex2Oat class
  // should not destruct the runtime in this case.
  if (!art::kIsDebugBuild && (RUNNING_ON_VALGRIND == 0)) {
    exit(result);
  }
  return result;
}


Constructor
Let's take a look at the following flowchart against the code.


The dex2oat logic is clear.
First, a workaround must be implemented on an arm. This is not related to the analysis and therefore you only need to have a basic idea about it.
Then, the following operations are performed:
1. Construct a dex2oat object.
2. Process CLI arguments.
3. Check whether files can be written.
4. Print CLI arguments.
5. Check whether dex2oat setup is complete.
6. Call CompileImage or CompileApp to perform processing, depending on whether the object to be processed is an image.
The code includes detailed comments that are quite understandable. The code is as follows:
static int dex2oat(int argc, char** argv) {
  b13564922();

  TimingLogger timings("compiler", false, false);

  Dex2Oat dex2oat(&timings);

  // Parse arguments. Argument mistakes will lead to exit(EXIT_FAILURE) in UsageError.
  dex2oat.ParseArgs(argc, argv);

  // Check early that the result of compilation can be written
  if (!dex2oat.OpenFile()) {
    return EXIT_FAILURE;
  }

  // Print the complete line when any of the following is true:
  //   1) Debug build
  //   2) Compiling an image
  //   3) Compiling with --host
  //   4) Compiling on the host (not a target build)
  // Otherwise, print a stripped command line.
  if (kIsDebugBuild || dex2oat.IsImage() || dex2oat.IsHost() || !kIsTargetBuild) {
    LOG(INFO) << CommandLine();
  } else {
    LOG(INFO) << StrippedCommandLine();
  }

  if (!dex2oat.Setup()) {
    dex2oat.EraseOatFile();
    return EXIT_FAILURE;
  }

  if (dex2oat.IsImage()) {
    return CompileImage(dex2oat);
  } else {
    return CompileApp(dex2oat);
  }
}
}  // namespace art


CompileApp
I will detail CLI argument processing later. Now, let's look at the core logic that involves CompileApp first.
As we can see, the core logic is implemented mainly for dex2oat.Compile encapsulation, followed by file writing and timekeeping.
static int CompileApp(Dex2Oat& dex2oat) {
  dex2oat.Compile();

  // Create the app oat.
  if (!dex2oat.CreateOatFile()) {
    dex2oat.EraseOatFile();
    return EXIT_FAILURE;
  }

  // Do not close the oat file here. We might haven gotten the output file by file descriptor,
  // which we would lose.
  if (!dex2oat.FlushOatFile()) {
    return EXIT_FAILURE;
  }

  // When given --host, finish early without stripping.
  if (dex2oat.IsHost()) {
    if (!dex2oat.FlushCloseOatFile()) {
      return EXIT_FAILURE;
    }

    dex2oat.DumpTiming();
    return EXIT_SUCCESS;
  }

  // Copy unstripped to stripped location, if necessary. This will implicitly flush & close the
  // unstripped version. If this is given, we expect to be able to open writable files by name.
  if (!dex2oat.CopyUnstrippedToStripped()) {
    return EXIT_FAILURE;
  }

  // Flush and close the file.
  if (!dex2oat.FlushCloseOatFile()) {
    return EXIT_FAILURE;
  }

  dex2oat.DumpTiming();
  return EXIT_SUCCESS;
}


CompileImage
CompileImage is similar to CompileApp.
static int CompileImage(Dex2Oat& dex2oat) {
  dex2oat.Compile();

  // Create the boot.oat.
  if (!dex2oat.CreateOatFile()) {
    dex2oat.EraseOatFile();
    return EXIT_FAILURE;
  }

  // Flush and close the boot.oat. We always expect the output file by name, and it will be
  // re-opened from the unstripped name.
  if (!dex2oat.FlushCloseOatFile()) {
    return EXIT_FAILURE;
  }

  // Creates the boot.art and patches the boot.oat.
  if (!dex2oat.HandleImage()) {
    return EXIT_FAILURE;
  }

  // When given --host, finish early without stripping.
  if (dex2oat.IsHost()) {
    dex2oat.DumpTiming();
    return EXIT_SUCCESS;
  }

  // Copy unstripped to stripped location, if necessary.
  if (!dex2oat.CopyUnstrippedToStripped()) {
    return EXIT_FAILURE;
  }

  // FlushClose again, as stripping might have re-opened the oat file.
  if (!dex2oat.FlushCloseOatFile()) {
    return EXIT_FAILURE;
  }

  dex2oat.DumpTiming();
  return EXIT_SUCCESS;
}


Compile
Java provides ClassLoader, which is not provided by many other compilation languages. Before compilation, ClassLoader must be preprocessed.
Then, a CompilerDriver object is created, and CompileAll of CompilerDriver is called to complete compilation.
// Create and invoke the compiler driver. This will compile all the dex files.
  void Compile() {
    TimingLogger::ScopedTiming t("dex2oat Compile", timings_);
    compiler_phases_timings_.reset(new CumulativeLogger("compilation times"));

    // Handle and ClassLoader creation needs to come after Runtime::Create
    jobject class_loader = nullptr;
    Thread* self = Thread::Current();
    if (!boot_image_option_.empty()) {
      ClassLinker* class_linker = Runtime::Current()->GetClassLinker();
      OpenClassPathFiles(runtime_->GetClassPathString(), dex_files_, &class_path_files_);
      ScopedObjectAccess soa(self);

      // Classpath: first the class-path given.
      std::vector<const DexFile*> class_path_files;
      for (auto& class_path_file : class_path_files_) {
        class_path_files.push_back(class_path_file.get());
      }

      // Store the classpath we have right now.
      key_value_store_->Put(OatHeader::kClassPathKey,
                            OatFile::EncodeDexFileDependencies(class_path_files));

      // Then the dex files we'll compile. Thus we'll resolve the class-path first.
      class_path_files.insert(class_path_files.end(), dex_files_.begin(), dex_files_.end());

      class_loader = class_linker->CreatePathClassLoader(self, class_path_files);
    }

    driver_ = new CompilerDriver(compiler_options_.get(),
                                 verification_results_,
                                 &method_inliner_map_,
                                 compiler_kind_,
                                 instruction_set_,
                                 instruction_set_features_.get(),
                                 image_,
                                 image_classes_.release(),
                                 compiled_classes_.release(),
                                 nullptr,
                                 thread_count_,
                                 dump_stats_,
                                 dump_passes_,
                                 dump_cfg_file_name_,
                                 compiler_phases_timings_.get(),
                                 swap_fd_,
                                 profile_file_);

    driver_->CompileAll(class_loader, dex_files_, timings_);
  }


Constructor of CompilerDriver
The core logic also focuses on compiler_ initialization.
Since the constructor involves many dex2oat CLI arguments, it is necessary to introduce them again, though all the arguments have been introduced before in Insight into dex2oat (1).
CompilerDriver::CompilerDriver(const CompilerOptions* compiler_options,
                               VerificationResults* verification_results,
                               DexFileToMethodInlinerMap* method_inliner_map,
                               Compiler::Kind compiler_kind,
                               InstructionSet instruction_set,
                               const InstructionSetFeatures* instruction_set_features,
                               bool image, std::unordered_set<std::string>* image_classes,
                               std::unordered_set<std::string>* compiled_classes,
                               std::unordered_set<std::string>* compiled_methods,
                               size_t thread_count, bool dump_stats, bool dump_passes,
                               const std::string& dump_cfg_file_name, CumulativeLogger* timer,
                               int swap_fd, const std::string& profile_file)
    : swap_space_(swap_fd == -1 ? nullptr : new SwapSpace(swap_fd, 10 * MB)),
      swap_space_allocator_(new SwapAllocator<void>(swap_space_.get())),
      profile_present_(false), compiler_options_(compiler_options),
      verification_results_(verification_results),
      method_inliner_map_(method_inliner_map),
      compiler_(Compiler::Create(this, compiler_kind)),
      compiler_kind_(compiler_kind),
      instruction_set_(instruction_set),
      instruction_set_features_(instruction_set_features),
      freezing_constructor_lock_("freezing constructor lock"),
      compiled_classes_lock_("compiled classes lock"),
      compiled_methods_lock_("compiled method lock"),
      compiled_methods_(MethodTable::key_compare()),
      non_relative_linker_patch_count_(0u),
      image_(image),
      image_classes_(image_classes),
      classes_to_compile_(compiled_classes),
      methods_to_compile_(compiled_methods),
      had_hard_verifier_failure_(false),
      thread_count_(thread_count),
      stats_(new AOTCompilationStats),
      dedupe_enabled_(true),
      dump_stats_(dump_stats),
      dump_passes_(dump_passes),
      dump_cfg_file_name_(dump_cfg_file_name),
      timings_logger_(timer),
      compiler_context_(nullptr),
      support_boot_image_fixup_(instruction_set != kMips && instruction_set != kMips64),
      dedupe_code_("dedupe code", *swap_space_allocator_),
      dedupe_src_mapping_table_("dedupe source mapping table", *swap_space_allocator_),
      dedupe_mapping_table_("dedupe mapping table", *swap_space_allocator_),
      dedupe_vmap_table_("dedupe vmap table", *swap_space_allocator_),
      dedupe_gc_map_("dedupe gc map", *swap_space_allocator_),
      dedupe_cfi_info_("dedupe cfi info", *swap_space_allocator_) {
  DCHECK(compiler_options_ != nullptr);
  DCHECK(verification_results_ != nullptr);
  DCHECK(method_inliner_map_ != nullptr);

  dex_to_dex_compiler_ = reinterpret_cast<DexToDexCompilerFn>(ArtCompileDEX);

  compiler_->Init();

  CHECK_EQ(image_, image_classes_.get() != nullptr);
...
}


CompilerDriver::CompileAll
CompilerDriver first creates a pool of compilation threads, which is a preferable step.
Compilation is divided into two steps:
• PreCompile
• Compile
void CompilerDriver::CompileAll(jobject class_loader,
                                const std::vector<const DexFile*>& dex_files,
                                TimingLogger* timings) {
  DCHECK(!Runtime::Current()->IsStarted());
  std::unique_ptr<ThreadPool> thread_pool(
      new ThreadPool("Compiler driver thread pool", thread_count_ - 1));
...
  PreCompile(class_loader, dex_files, thread_pool.get(), timings);
  Compile(class_loader, dex_files, thread_pool.get(), timings);
...
}


CompilerDriver::PreCompile
In the PreCompile step, the following two major operations are performed:
• Verification
• Class initialization
I will skip the operation of checking whether verification is required, so the processing logic will seem clearer to you.
void CompilerDriver::PreCompile(jobject class_loader, const std::vector<const DexFile*>& dex_files,
                                ThreadPool* thread_pool, TimingLogger* timings) {
...

  Verify(class_loader, dex_files, thread_pool, timings);
...
  if (had_hard_verifier_failure_ && GetCompilerOptions().AbortOnHardVerifierFailure()) {
    LOG(FATAL) << "Had a hard failure verifying all classes, and was asked to abort in such "
               << "situations. Please check the log.";
  }

  InitializeClasses(class_loader, dex_files, thread_pool, timings);
...
}


CompilerDriver::Compile
CompileDexFile is called to compile each dex file.
void CompilerDriver::Compile(jobject class_loader, const std::vector<const DexFile*>& dex_files,
                             ThreadPool* thread_pool, TimingLogger* timings) {
  for (size_t i = 0; i != dex_files.size(); ++i) {
    const DexFile* dex_file = dex_files;
    CHECK(dex_file != nullptr);
    CompileDexFile(class_loader, *dex_file, dex_files, thread_pool, timings);
  }
...
}


CompilerDriver::CompileDexFile
The Compile function complies multiple dex files individually, while the CompileDexFile function divides dex files by class and calls CompileClass to compile the files for each class.
void CompilerDriver::CompileDexFile(jobject class_loader, const DexFile& dex_file,
                                    const std::vector<const DexFile*>& dex_files,
                                    ThreadPool* thread_pool, TimingLogger* timings) {
  TimingLogger::ScopedTiming t("Compile Dex File", timings);
  ParallelCompilationManager context(Runtime::Current()->GetClassLinker(), class_loader, this,
                                     &dex_file, dex_files, thread_pool);
  context.ForAll(0, dex_file.NumClassDefs(), CompilerDriver::CompileClass, thread_count_);
}


Summary
Take a look at the procedure again for a better understanding.
Guest