[MLIR][LLVM] Move the LLVM inliner interface into a separate file.

clang/docs/UsersManual.rst

@@ -4487,3 +4487,126 @@ If the user is using the static CRT (``/MT``), then different runtimes are used
 to produce DLLs and EXEs. To link a DLL, pass
 ``clang_rt.asan_dll_thunk-x86_64.lib``. To link an EXE, pass
 ``-wholearchive:clang_rt.asan-x86_64.lib``.
+
+Windows System Headers and Library Lookup
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+clang-cl uses a set of different approaches to locate the right system libraries
+to link against when building code.  The Windows environment uses libraries from
+three distinct sources:
+
+1. Windows SDK
+2. UCRT (Universal C Runtime)
+3. Visual C++ Tools (VCRuntime)
+
+The Windows SDK provides the import libraries and headers required to build
+programs against the Windows system packages.  Underlying the Windows SDK is the
+UCRT, the universal C runtime.
+
+This difference is best illustrated by the various headers that one would find
+in the different categories.  The WinSDK would contain headers such as
+`WinSock2.h` which is part of the Windows API surface, providing the Windows
+socketing interfaces for networking.  UCRT provides the C library headers,
+including e.g. `stdio.h`.  Finally, the Visual C++ tools provides the underlying
+Visual C++ Runtime headers such as `stdint.h` or `crtdefs.h`.
+
+There are various controls that allow the user control over where clang-cl will
+locate these headers.  The default behaviour for the Windows SDK and UCRT is as
+follows:
+
+1. Consult the command line.
+
+    Anything the user specifies is always given precedence.  The following
+    extensions are part of the clang-cl toolset:
+
+    - `/winsysroot:`
+
+    The `/winsysroot:` is used as an equivalent to `-sysroot` on Unix
+    environments.  It allows the control of an alternate location to be treated
+    as a system root.  When specified, it will be used as the root where the
+    `Windows Kits` is located.
+
+    - `/winsdkversion:`
+    - `/winsdkdir:`
+
+    If `/winsysroot:` is not specified, the `/winsdkdir:` argument is consulted
+    as a location to identify where the Windows SDK is located.  Contrary to
+    `/winsysroot:`, `/winsdkdir:` is expected to be the complete path rather
+    than a root to locate `Windows Kits`.
+
+    The `/winsdkversion:` flag allows the user to specify a version identifier
+    for the SDK to prefer.  When this is specified, no additional validation is
+    performed and this version is preferred.  If the version is not specified,
+    the highest detected version number will be used.
+
+2. Consult the environment.
+
+    TODO: This is not yet implemented.
+
+    This will consult the environment variables:
+
+    - `WindowsSdkDir`
+    - `UCRTVersion`
+
+3. Fallback to the registry.
+
+    If no arguments are used to indicate where the SDK is present, and the
+    compiler is running on Windows, the registry is consulted to locate the
+    installation.
+
+The Visual C++ Toolset has a slightly more elaborate mechanism for detection.
+
+1. Consult the command line.
+
+    - `/winsysroot:`
+
+    The `/winsysroot:` is used as an equivalent to `-sysroot` on Unix
+    environments.  It allows the control of an alternate location to be treated
+    as a system root.  When specified, it will be used as the root where the
+    `VC` directory is located.
+
+    - `/vctoolsdir:`
+    - `/vctoolsversion:`
+
+    If `/winsysroot:` is not specified, the `/vctoolsdir:` argument is consulted
+    as a location to identify where the Visual C++ Tools are located.  If
+    `/vctoolsversion:` is specified, that version is preferred, otherwise, the
+    highest version detected is used.
+
+2. Consult the environment.
+
+    - `/external:[VARIABLE]`
+
+      This specifies a user identified environment variable which is treated as
+      a path delimiter (`;`) separated list of paths to map into `-imsvc`
+      arguments which are treated as `-isystem`.
+
+    - `INCLUDE` and `EXTERNAL_INCLUDE`
+
+      The path delimiter (`;`) separated list of paths will be mapped to
+      `-imsvc` arguments which are treated as `-isystem`.
+
+    - `LIB` (indirectly)
+
+      The linker `link.exe` or `lld-link.exe` will honour the environment
+      variable `LIB` which is a path delimiter (`;`) set of paths to consult for
+      the import libraries to use when linking the final target.
+
+    The following environment variables will be consulted and used to form paths
+    to validate and load content from as appropriate:
+
+      - `VCToolsInstallDir`
+      - `VCINSTALLDIR`
+      - `Path`
+
+3. Consult `ISetupConfiguration` [Windows Only]
+
+    Assuming that the toolchain is built with `USE_MSVC_SETUP_API` defined and
+    is running on Windows, the Visual Studio COM interface `ISetupConfiguration`
+    will be used to locate the installation of the MSVC toolset.
+
+4. Fallback to the registry [DEPRECATED]
+
+    The registry information is used to help locate the installation as a final
+    fallback.  This is only possible for pre-VS2017 installations and is
+    considered deprecated.

clang/include/clang/Analysis/FlowSensitive/DataflowAnalysisContext.h

@@ -264,7 +264,8 @@ class DataflowAnalysisContext {
   /// `Val2` imposed by the flow condition.
   bool equivalentBoolValues(BoolValue &Val1, BoolValue &Val2);
 
-  LLVM_DUMP_METHOD void dumpFlowCondition(AtomicBoolValue &Token);
+  LLVM_DUMP_METHOD void dumpFlowCondition(AtomicBoolValue &Token,
+                                          llvm::raw_ostream &OS = llvm::dbgs());
 
   /// Returns the `ControlFlowContext` registered for `F`, if any. Otherwise,
   /// returns null.

clang/include/clang/Driver/Options.td

@@ -5080,6 +5080,10 @@ def flang_experimental_exec : Flag<["-"], "flang-experimental-exec">,
   Flags<[FlangOption, FlangOnlyOption, NoXarchOption, HelpHidden]>,
   HelpText<"Enable support for generating executables (experimental)">;
 
+def flang_experimental_hlfir : Flag<["-"], "flang-experimental-hlfir">,
+  Flags<[FlangOption, FC1Option, FlangOnlyOption, NoXarchOption, HelpHidden]>,
+  HelpText<"Use HLFIR lowering (experimental)">;
+
 //===----------------------------------------------------------------------===//
 // FLangOption + CoreOption + NoXarchOption
 //===----------------------------------------------------------------------===//

clang/lib/Analysis/FlowSensitive/DataflowAnalysisContext.cpp

@@ -342,15 +342,16 @@ BoolValue &DataflowAnalysisContext::buildAndSubstituteFlowConditionWithCache(
   return substituteBoolValue(*ConstraintsIt->second, SubstitutionsCache);
 }
 
-void DataflowAnalysisContext::dumpFlowCondition(AtomicBoolValue &Token) {
+void DataflowAnalysisContext::dumpFlowCondition(AtomicBoolValue &Token,
+                                                llvm::raw_ostream &OS) {
   llvm::DenseSet<BoolValue *> Constraints = {&Token};
   llvm::DenseSet<AtomicBoolValue *> VisitedTokens;
   addTransitiveFlowConditionConstraints(Token, Constraints, VisitedTokens);
 
   llvm::DenseMap<const AtomicBoolValue *, std::string> AtomNames = {
       {&getBoolLiteralValue(false), "False"},
       {&getBoolLiteralValue(true), "True"}};
-  llvm::dbgs() << debugString(Constraints, AtomNames);
+  OS << debugString(Constraints, AtomNames);
 }
 
 const ControlFlowContext *

clang/lib/Analysis/FlowSensitive/DataflowEnvironment.cpp

@@ -812,7 +812,7 @@ void Environment::dump(raw_ostream &OS) const {
   }
 
   OS << "FlowConditionToken:\n";
-  DACtx->dumpFlowCondition(*FlowConditionToken);
+  DACtx->dumpFlowCondition(*FlowConditionToken, OS);
 }
 
 void Environment::dump() const {

clang/lib/Driver/ToolChains/Flang.cpp

@@ -65,6 +65,9 @@ void Flang::addOtherOptions(const ArgList &Args, ArgStringList &CmdArgs) const {
   if (stackArrays &&
       !stackArrays->getOption().matches(options::OPT_fno_stack_arrays))
     CmdArgs.push_back("-fstack-arrays");
+
+  if (Args.hasArg(options::OPT_flang_experimental_hlfir))
+    CmdArgs.push_back("-flang-experimental-hlfir");
 }
 
 void Flang::addPicOptions(const ArgList &Args, ArgStringList &CmdArgs) const {

cross-project-tests/debuginfo-tests/dexter/dex/debugger/DebuggerControllers/ConditionalController.py

@@ -16,6 +16,7 @@
 from dex.debugger.DebuggerControllers.DebuggerControllerBase import DebuggerControllerBase
 from dex.debugger.DebuggerBase import DebuggerBase
 from dex.utils.Exceptions import DebuggerException
+from dex.utils.Timeout import Timeout
 
 
 class BreakpointRange:
@@ -140,10 +141,26 @@ def _run_debugger_custom(self, cmdline):
         time.sleep(self._pause_between_steps)
 
         exit_desired = False
+        timed_out = False
+        total_timeout = Timeout(self.context.options.timeout_total)
 
         while not self.debugger.is_finished:
-            while self.debugger.is_running:
-                pass
+
+            breakpoint_timeout = Timeout(self.context.options.timeout_breakpoint)
+            while self.debugger.is_running and not timed_out:
+                # Check to see whether we've timed out while we're waiting.
+                if total_timeout.timed_out():
+                    self.context.logger.error('Debugger session has been '
+                        f'running for {total_timeout.elapsed}s, timeout reached!')
+                    timed_out = True
+                if breakpoint_timeout.timed_out():
+                    self.context.logger.error(f'Debugger session has not '
+                        f'hit a breakpoint for {breakpoint_timeout.elapsed}s, timeout '
+                        'reached!')
+                    timed_out = True
+
+            if timed_out:
+                break
 
             step_info = self.debugger.get_step_info(self._watches, self._step_index)
             if step_info.current_frame:

cross-project-tests/debuginfo-tests/dexter/dex/debugger/DebuggerControllers/DefaultController.py

@@ -13,6 +13,7 @@
 from dex.debugger.DebuggerControllers.DebuggerControllerBase import DebuggerControllerBase
 from dex.debugger.DebuggerControllers.ControllerHelpers import in_source_file, update_step_watches
 from dex.utils.Exceptions import DebuggerException, LoadDebuggerException
+from dex.utils.Timeout import Timeout
 
 class EarlyExitCondition(object):
     def __init__(self, on_line, hit_count, expression, values):
@@ -81,12 +82,25 @@ def _run_debugger_custom(self, cmdline):
             self.watches.update(command_obj.get_watches())
         early_exit_conditions = self._get_early_exit_conditions()
 
+        timed_out = False
+        total_timeout = Timeout(self.context.options.timeout_total)
         max_steps = self.context.options.max_steps
         for _ in range(max_steps):
-            while self.debugger.is_running:
-                pass
 
-            if self.debugger.is_finished:
+            breakpoint_timeout = Timeout(self.context.options.timeout_breakpoint)
+            while self.debugger.is_running and not timed_out:
+                # Check to see whether we've timed out while we're waiting.
+                if total_timeout.timed_out():
+                    self.context.logger.error('Debugger session has been '
+                        f'running for {total_timeout.elapsed}s, timeout reached!')
+                    timed_out = True
+                if breakpoint_timeout.timed_out():
+                    self.context.logger.error(f'Debugger session has not '
+                        f'hit a breakpoint for {breakpoint_timeout.elapsed}s, timeout '
+                        'reached!')
+                    timed_out = True
+
+            if timed_out or self.debugger.is_finished:
                 break
 
             self.step_index += 1

cross-project-tests/debuginfo-tests/dexter/dex/debugger/Debuggers.py

@@ -121,7 +121,21 @@ def add_debugger_tool_arguments(parser, context, defaults):
         default='',
         help='command line arguments for the test program, in addition to any '
              'provided by DexCommandLine')
-
+    parser.add_argument(
+        '--timeout-total',
+        metavar='<seconds>',
+        type=float,
+        default=0.0,
+        help='if >0, debugger session will automatically exit after '
+             'running for <timeout-total> seconds')
+    parser.add_argument(
+        '--timeout-breakpoint',
+        metavar='<seconds>',
+        type=float,
+        default=0.0,
+        help='if >0, debugger session will automatically exit after '
+             'waiting <timeout-breakpoint> seconds without hitting a '
+             'breakpoint')
 
 def handle_debugger_tool_base_options(context, defaults):  # noqa
     options = context.options

cross-project-tests/debuginfo-tests/dexter/dex/utils/Timeout.py

@@ -0,0 +1,31 @@
+# DExTer : Debugging Experience Tester
+# ~~~~~~   ~         ~~         ~   ~~
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+"""Utility class to check for timeouts. Timer starts when the object is initialized,
+and can be checked by calling timed_out(). Passing a timeout value of 0.0 or less
+means a timeout will never be triggered, i.e. timed_out() will always return False.
+"""
+
+import time
+
+class Timeout(object):
+
+    def __init__(self, duration: float):
+        self.start = self.now
+        self.duration = duration
+
+    def timed_out(self):
+        if self.duration <= 0.0:
+            return False
+        return self.elapsed > self.duration
+
+    @property
+    def elapsed(self):
+        return self.now - self.start
+
+    @property
+    def now(self):
+        return time.time()

flang/include/flang/Tools/CLOptions.inc

@@ -14,6 +14,7 @@
 #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
 #include "mlir/Transforms/Passes.h"
 #include "flang/Optimizer/CodeGen/CodeGen.h"
+#include "flang/Optimizer/HLFIR/Passes.h"
 #include "flang/Optimizer/Transforms/Passes.h"
 #include "llvm/Passes/OptimizationLevel.h"
 #include "llvm/Support/CommandLine.h"
@@ -72,7 +73,8 @@ DisableOption(BoxedProcedureRewrite, "boxed-procedure-rewrite",
     "rewrite boxed procedures");
 #endif
 
-DisableOption(ExternalNameConversion, "external-name-interop", "convert names with external convention");
+DisableOption(ExternalNameConversion, "external-name-interop",
+    "convert names with external convention");
 
 /// Generic for adding a pass to the pass manager if it is not disabled.
 template <typename F>
@@ -211,15 +213,28 @@ inline void createDefaultFIROptimizerPassPipeline(mlir::PassManager &pm,
   pm.addPass(mlir::createCSEPass());
 }
 
+/// Create a pass pipeline for lowering from HLFIR to FIR
+///
+/// \param pm - MLIR pass manager that will hold the pipeline definition
+/// \param optLevel - optimization level used for creating FIR optimization
+///   passes pipeline
+inline void createHLFIRToFIRPassPipeline(
+    mlir::PassManager &pm, llvm::OptimizationLevel optLevel = defaultOptLevel) {
+  if (optLevel.isOptimizingForSpeed())
+    pm.addPass(mlir::createCanonicalizerPass());
+  pm.addPass(hlfir::createLowerHLFIRIntrinsicsPass());
+  pm.addPass(hlfir::createBufferizeHLFIRPass());
+  pm.addPass(hlfir::createConvertHLFIRtoFIRPass());
+}
+
 #if !defined(FLANG_EXCLUDE_CODEGEN)
 inline void createDefaultFIRCodeGenPassPipeline(mlir::PassManager &pm,
     llvm::OptimizationLevel optLevel = defaultOptLevel,
     bool underscoring = true) {
   fir::addBoxedProcedurePass(pm);
   pm.addNestedPass<mlir::func::FuncOp>(
       fir::createAbstractResultOnFuncOptPass());
-  pm.addNestedPass<fir::GlobalOp>(
-      fir::createAbstractResultOnGlobalOptPass());
+  pm.addNestedPass<fir::GlobalOp>(fir::createAbstractResultOnGlobalOptPass());
   fir::addCodeGenRewritePass(pm);
   fir::addTargetRewritePass(pm);
   fir::addExternalNameConversionPass(pm, underscoring);
@@ -234,6 +249,8 @@ inline void createDefaultFIRCodeGenPassPipeline(mlir::PassManager &pm,
 inline void createMLIRToLLVMPassPipeline(mlir::PassManager &pm,
     llvm::OptimizationLevel optLevel = defaultOptLevel,
     bool stackArrays = false, bool underscoring = true) {
+  fir::createHLFIRToFIRPassPipeline(pm, optLevel);
+
   // Add default optimizer pass pipeline.
   fir::createDefaultFIROptimizerPassPipeline(pm, optLevel, stackArrays);
 

flang/lib/Frontend/CompilerInvocation.cpp

@@ -818,6 +818,11 @@ bool CompilerInvocation::createFromArgs(
     success = false;
   }
 
+  // -flang-experimental-hlfir
+  if (args.hasArg(clang::driver::options::OPT_flang_experimental_hlfir)) {
+    res.loweringOpts.setLowerToHighLevelFIR(true);
+  }
+
   success &= parseFrontendArgs(res.getFrontendOpts(), args, diags);
   parseTargetArgs(res.getTargetOpts(), args);
   parsePreprocessorArgs(res.getPreprocessorOpts(), args);

flang/test/Driver/driver-help-hidden.f90

@@ -41,6 +41,8 @@
 ! CHECK-NEXT:                        Specify where to find the compiled intrinsic modules
 ! CHECK-NEXT: -flang-experimental-exec
 ! CHECK-NEXT:                        Enable support for generating executables (experimental)
+! CHECK-NEXT: -flang-experimental-hlfir
+! CHECK-NEXT:                        Use HLFIR lowering (experimental)
 ! CHECK-NEXT: -flarge-sizes          Use INTEGER(KIND=8) for the result type in size-related intrinsics
 ! CHECK-NEXT: -flogical-abbreviations Enable logical abbreviations
 ! CHECK-NEXT: -flto=<value> Set LTO mode