|
| 1 | + |
| 2 | +# Inference |
| 3 | + |
| 4 | +## Debugging inference.jl |
| 5 | + |
| 6 | +You can start a Julia session, edit `inference.jl` (for example to |
| 7 | +insert `print` statements), and then replace `Core.Inference` in your |
| 8 | +running session by navigating to `base/` and executing |
| 9 | +`include("coreimg.jl")`. This trick typically leads to much faster |
| 10 | +development than if you rebuild Julia for each change. |
| 11 | + |
| 12 | +A convenient entry point into inference is `typeinf_code`. Here's a |
| 13 | +demo running inference on `convert(Int, UInt(1))`: |
| 14 | + |
| 15 | +```julia |
| 16 | +# Get the method |
| 17 | +atypes = Tuple{Type{Int}, UInt} # argument types |
| 18 | +mths = methods(convert, atypes) # worth checking that there is only one |
| 19 | +m = first(mths) |
| 20 | + |
| 21 | +# Create variables needed to call `typeinf_code` |
| 22 | +params = Core.Inference.InferenceParams(typemax(UInt)) # parameter is the world age, |
| 23 | + # typemax(UInt) -> most recent |
| 24 | +sparams = Core.svec() # this particular method doesn't have type-parameters |
| 25 | +optimize = true # run all inference optimizations |
| 26 | +cached = false # force inference to happen (do not use cached results) |
| 27 | +Core.Inference.typeinf_code(m, atypes, sparams, optimize, cached, params) |
| 28 | +``` |
| 29 | + |
| 30 | +If your debugging adventures require a `MethodInstance`, you can look it up by |
| 31 | +calling `Core.Inference.code_for_method` using many of the variables above. |
| 32 | +A `CodeInfo` object may be obtained with |
| 33 | +```julia |
| 34 | +# Returns the CodeInfo object for `convert(Int, ::UInt)`: |
| 35 | +ci = (@code_typed convert(Int, UInt(1)))[1] |
| 36 | +``` |
| 37 | + |
| 38 | +## The inlining algorithm (inline_worthy) |
| 39 | + |
| 40 | +Much of the hardest work for inlining runs in |
| 41 | +`inlining_pass`. However, if your question is "why didn't my function |
| 42 | +inline?" then you will most likely be interested in `isinlineable` and |
| 43 | +its primary callee, `inline_worthy`. `isinlineable` handles a number |
| 44 | +of special cases (e.g., critical functions like `next` and `done`, |
| 45 | +incorporating a bonus for functions that return tuples, etc.). The |
| 46 | +main decision-making happens in `inline_worthy`, which returns `true` |
| 47 | +if the function should be inlined. |
| 48 | + |
| 49 | +`inline_worthy` implements a cost-model, where "cheap" functions get |
| 50 | +inlined; more specifically, we inline functions if their anticipated |
| 51 | +run-time is not large compared to the time it would take to |
| 52 | +[issue a call](https://en.wikipedia.org/wiki/Calling_convention) to |
| 53 | +them if they were not inlined. The cost-model is extremely simple and |
| 54 | +ignores many important details: for example, all `for` loops are |
| 55 | +analyzed as if they will be executed once, and the cost of an |
| 56 | +`if...else...end` includes the summed cost of all branches. It's also |
| 57 | +worth acknowledging that we currently lack a suite of functions |
| 58 | +suitable for testing how well the cost model predicts the actual |
| 59 | +run-time cost, although |
| 60 | +[BaseBenchmarks](https://github.com/JuliaCI/BaseBenchmarks.jl) |
| 61 | +provides a great deal of indirect information about the successes and |
| 62 | +failures of any modification to the inlining algorithm. |
| 63 | + |
| 64 | +The foundation of the cost-model is a lookup table, implemented in |
| 65 | +`add_tfunc` and its callers, that assigns an estimated cost (measured |
| 66 | +in CPU cycles) to each of Julia's intrinsic functions. These costs are |
| 67 | +based on |
| 68 | +[standard ranges for common architectures](http://ithare.com/wp-content/uploads/part101_infographics_v08.png) |
| 69 | +(see |
| 70 | +[Agner Fog's analysis](http://www.agner.org/optimize/instruction_tables.pdf) |
| 71 | +for more detail). |
| 72 | + |
| 73 | +We supplement this low-level lookup table with a number of special |
| 74 | +cases. For example, an `:invoke` expression (a call for which all |
| 75 | +input and output types were inferred in advance) is assigned a fixed |
| 76 | +cost (currently 20 cycles). In contrast, a `:call` expression, for |
| 77 | +other than intrinsics/builtins, indicates that the call will require |
| 78 | +dynamic dispatch, in which case we assign a cost set by |
| 79 | +`InferenceParams.inline_nonleaf_penalty` (currently set at 1000). Note |
| 80 | +that this is not a "first-principles" estimate of the raw cost of |
| 81 | +dynamic dispatch, but a mere heuristic indicating that dynamic |
| 82 | +dispatch is extremely expensive. |
| 83 | + |
| 84 | +Each statement gets analyzed for its total cost in a function called |
| 85 | +`statement_cost`. You can run this yourself by following this example: |
| 86 | + |
| 87 | +```julia |
| 88 | +params = Core.Inference.InferenceParams(typemax(UInt)) |
| 89 | +# Get the CodeInfo object |
| 90 | +ci = (@code_typed fill(3, (5, 5)))[1] # we'll try this on the code for `fill(3, (5, 5))` |
| 91 | +# Calculate cost of each statement |
| 92 | +cost(stmt) = Core.Inference.statement_cost(stmt, ci, Base, params) |
| 93 | +cst = map(cost, ci.code) |
| 94 | +``` |
| 95 | + |
| 96 | +The output is a `Vector{Int}` holding the estimated cost of each |
| 97 | +statement in `ci.code`. Note that `ci` includes the consequences of |
| 98 | +inlining callees, and consequently the costs do too. |
0 commit comments