TVM 0.5 Release Note

[ZihengJiang]

The release is managed by the TVM PMC, moderators for this release are @ZihengJiang and @yzhliu

The community welcomes new reviewers @nishi-t @were @siju-samuel @jroesch @xqdan @zhiics @grwlf @ajtulloch @vinx13 @junrushao1994 @FrozenGene @liangfu , new committers @srkreddy1238 @eqy @masahi @nhynes @phisiart @merrymercy @Laurawly @adityaatluri @Huyuwei

Change List

New Features

• Fully featured 8-bit network support
  • 8bit quantizer
  • Arbitrary bits quantization algorithm
  • Intel cpu support
• NVidia GPU 8-bit kernel
  • int8 gemm recipe
  • int8 conv2d
  • Autotvm integration
• Automated tuning and scheduling
  • AutoTVM optimizations for mobile GPUs
  • AutoTVM optimizations for CUDA
  • AutoTVM optimizations for x86
• Initial release of the differentiable programming IR, Relay
  • Generic & informative Relay error reporting [Relay] Add generic & informative Relay error reporting #2408
  • Relay IR text format support [Relay][RFC] Relay IR Text Format #1781
  • Support control flows
  • A Normal Form Canonicalization [Relay] A Normal Form Canonicalization #2251
  • Type system support
  • End to end compilation
    • Frontend support: Caffe2 [Relay][Frontend] Caffe2 Support #2507 , CoreML [Relay][Frontend] CoreML Support #2476 , Keras [Relay][Keras] force const dtype to be float32 #2376 , MXNet [RELAY][FRONTEND] Initial MXNet frontend support. #2163 , ONNX, TFLite [TFLite] Support TFLite FP32 Relay frontend. #2365
    • Operator coverage [RELAY][OP] Relay Operator Sprint #1799 [RELAY][OP] Relay Op Sprint (Part 2) #2051
  • FoldScaleAxis [RELAY][PASS] FoldScaleAxis Forward #2020
  • SimplifyInference [RELAY/PASS] Simplify inference. #2033
  • CombineParallelConv2D [RELAY][PASS] CombineParallelConv2D #2089
  • InstrumentBoundCheckers pass [PASS] InstrumentBoundCheckers pass #2079
  • Bind & FoldConstant [RELAY][PASS] Bind, FoldConstant #2100
  • Alter Op Layout [Relay] Alter Op Layout #2150
  • General OpFusion [RELAY][PASS] General OpFusion. #2090
• CodeGen
  • Gcc / g++ compatible C code generator for TVM [BACKEND][CODEGEN] C codegen with tests #2161
  • Device type annotation for heterogeneous compilation [relay][pass] Annotation for heterogeneous compilation #2361
  • Cache packed func ptr, lift alloca [CODEGEN][LLVM] Cache packed func ptr, lift alloca #2070
  • Generalize compute to tensor region [LANG] Generalize compute to tensor region #1476
• Runtime
  • Relay interpreter and compiler [RELAY][COMPILER] Initial Relay interpreter and compiler for TVM runtime system. #1954
  • Heterogeneous runtime Heterogeneous Runtime #1695
  • Language bindings: Golang runtime [RUNTIME][GOLANG] TVM runtime for golang v0.1 #1470 , Rust runtime Add rust runtime #1597
  • Add min_repeat_ms to time_evaluator [RUNTIME] Add min_repeat_ms to time_evaluator #2200
  • Bundled interpreter demonstration [Apps] [RFC] Bundled interpreter demonstration #2297
  • Enable PlanMemory in the graph runtime [RELAY][BACKEND] Enable PlanMemory in the graph runtime. #2120
• Language Binding
  • Rust frontend [RUST][FRONTEND] Add rust frontend v0.1 #2292
• VTA
  • Improved RPC for VTA [VTA] Improved RPC for VTA #2043
• Hybrid python programming model
  • Support for scheduling [Hybrid Script] Supporting scheduling hybrid script #2416
  • Support for Inter-function call [Hybrid Script] Inter-function call supported! #2287
  • Backend support [Hybrid script] Backend support #2477
• TOP
  • Initial support for sparse tensor computation
  • Improve ARM CPU depthwise convolution performance [ARM][Performance] Improve ARM CPU depthwise convolution performance #2345
  • Port winograd ops to relay [RELAY] Port winograd ops to relay #2356
• Tutorials and docs
  • Relay language docs [Relay][Docs] Relay Language Reference #2232
  • Tutorials on how to use SGX backend
  • How to write a pass in python
  • General lowering flow of TVM
  • How to do tensorize
  • TFLite frontend tutorial [Doc] TFLite frontend tutorial #2508
  • Keras seq2seq model for translation tutorial [Tutorial]NLP Sequence to sequence model for translation #1815
  • Committer guide and tips [DOCS][COMMUNITY] Committer guide and tips #2468
  • Code review guideline on API designs [DOCS][COMMUNITY] Improve code review guideline on API designs #2459

Bug-fixes

• Fix bug of ignoring user-defined shape in Tensorflow frontend. [NNVM][TENSORFLOW] bugfix. #2444
• Prevent generating 0-dim tensors in Tensorflow frontend. [FRONTEND][TENSORFLOW] Bugfix #2326
• Fix the bug of incorrect inferred layout when registered alter_layout function returns None. [Bugfix] Recover original layout when alter_layout function return None #2101
• Fix the return type of read character from a buffer. [BUGFIX] GetChar() in base64.h should return int, not char #2255
• Fix subgraph loading from JSON. Fix load subgraph from json #1980
• Fix incorrect value index in hybrid script. [BUGFIX] [Hybrid Script] fix in-correct value index in hybrid script #2268
• Fix out-of-bound access when buffer.access_ptr contains non-zero extent. Fix non-zero extent of access_ptr out of range #1939
• Add checker for vectorizing over reduce-axis. Vectorizing a reduce_axis does not raise an error #1795
• Bug fix for fusing injective operation and conv2d. [TOPI, CUDA] Bug fix properly bind gpu threads to injective op #1603 [NNVM] Bug fix Prevent fusing convolution with injective op  #1608 [NNVM] Bugfix operator fusion for residual block with layout transform #1760 [NNVM] Bug fix Add missing check when deciding conv and injective ops are in the same group #1622
• Fix bilinear mode in topi::nn::upsampling. Bugfix #1692. Constant folding and result comparision allowance (atol) #1708
• Fix the bug with which the Else statement disappears after inject virtual thread pass. Fix a bug in inject-virtual-thread #2039
• Preventing negation of an undefined Expr in some cases. [TVM] Fix negating undefined in DetectLinearEquation #1816
• Fix CUDA path lookup when multiple CUDA versions are installed. [BUG] Fix incorrect libcuda.so found by cmake when multiple versions of CUDA exist #1788
• Fix bugs for SSD sorting and multbox detection. Fixed bugs for SSD sorting and multbox detection #1578
• Fix incorrect elem_offset calculation in Buffer. fix buffer elem_offset calculation #1762
• Fix rewrite_unsafe_select pass so that LLVM CodeGen generates br instruction only when the type is i1. [LLVM CodeGen] Solve LLVM CodeGen br instruction accept not-i1 type issue #2381
• Fix incorrect intermediate results in debug mode. [BugFix] Copy intermediate result in debug runtime #2520
• Fix for SSD full GPU support and SSD tutorial update. [BugFix] SSD fully supported on GPUs, updated deploy_ssd tutorial #2510
• Fix LLVM CodeGen br instruction accept not-i1 type issue. [LLVM CodeGen] Solve LLVM CodeGen br instruction accept not-i1 type issue #2381
• Fixed constant folding for mod operation in CanonicalSimplify. [arith] fixed constant folding for mod operation in CanonicalSimplify #2487
• Fix missing reduction init predicates. missing predicates in split loops #2249
• Fix for quantize. [BUGFIX] Fix for quantize. #2573
• fix storage_rewrite bug when input is big. [TVM][Bugfix] fix storage_rewrite bug when input is big #2580

Contributors

Code reviewers

• @tqchen
• @liangfu quantization, relay, topi, frontend
• @zhiics relay, runtime, frontend
• @nhynes quantization, rust
• @Huyuwei frontend
• @yzhliu relay, frontend, perf
• @xqdan hybrid script, tvm/lang
• @ZihengJiang relay
• @vinx13 relay/pass, topi
• @masahi relay/pass, frontend, doc, topi
• @grwlf frontend, topi, relay, quantization
• @tmoreau89 vta, relay, backend, runtime
• @kazum frontend
• @nishi-t frontend, topi
• @PariksheetPinjari909 frontend
• @jroesch relay, frontend, doc
• @srkreddy1238 relay/op, frontend
• @siju-samuel relay/op, frontend
• @junrushao1994 relay
• @icemelon9 relay, perf, tvm/lang, codegen
• @ajtulloch relay, frontend
• @alex-weaver relay
• @kevinthesun hybrid script, topi, relay
• @Laurawly topi
• @were hybrid script, topi
• @FrozenGene frontend, topi, relay/pass
• @eqy relay, topi, runtime, rust
• @zhreshold frontend, relay/op
• @merrymercy relay/op, topi, runtime, frontend
• @derisavi-huawei symbolic integers

Code contributions

• @tqchen tvm
• @vinx13 relay/pass, topi
• @siju-samuel topi, relay/op
• @merrymercy autotvm, topi, relay/pass
• @srkreddy1238 relay/op, frontend/tf
• @MarisaKirisame relay
• @slyubomirsky relay, docs
• @jroesch relay
• @nhynes rust
• @wweic docs, relay/pass
• @yzhliu perf, frontend
• @zhiics relay/pass, relay/op, runtime
• @were hybrid script
• @icemelon9 perf, relay/pass, relay/op
• @joshpoll relay, docs
• @sgrechanik-h codegen
• @kazum frontend/keras, topi
• @masahi relay/op, docs
• @FrozenGene perf, frontend/tf
• @liangdzou docs
• @junrushao1994 relay/op
• @eqy autotvm, runtime
• @apivovarov docs
• @ajtulloch runtime, nnpack
• @kevinthesun relay/op, perf
• @ZihengJiang relay/pass, quantization
• @hlu1 nnpack, frontend/caffe2
• @lixiaoquan nnvm
• @imorinaga frontend/mxnet
• @liangfu topi, docs
• @xqdan codegen
• @PariksheetPinjari909 frontend/darknet
• @alexeyr frontend/tensorflow
• @Rasterer topi
• @yangchen-MS codegen
• @anijain2305 relay/op
• @grwlf topi
• @Huyuwei topi, frontend/keras
• @denis0x0D runtime/trace, relay/pass
• @mutinifni codegen
• @derisavi relay/pass
• @tmoreau89 vta
• @Laurawly topi, perf
• @zhreshold frontend, topi
• @kun-zh codegen
• @reminisce relay/op
• @ehsanmok rust
• @cnuernber perf
• @cowanmeg topi, codegen
• @yuruofeifei topi

[ZihengJiang]

Thanks to everyone who have pushed to last release cycle in the past months. We would like to propose the release of v0.5 on Jan 28th.

We encourage everyone in the community to put their weights to review and vote the release. @dmlc/tvm-team

Please reply this thread on

• Things that we missed in the release note
• Bugfixes that need to be included in this release

[liangfu]

We have brought the initial support for sparse tensor computation in TOPI (see #1289), though it would be migrated to use hybrid script instead of IR builder in the future.

[ehsanmok]

Hoping to make the rust frontend ready for this release.

[FrozenGene]

We have new frontend TFLite support (see #2365 ).

[yzhliu]

Pending PR for this release:

Required: #2507 #2476
Good to have: #2508 #2493 #2510 #2292

@ehsanmok as we're approaching the due, do we have chance to get the rust frontend in?

[FrozenGene]

BugFix:
[LLVM CodeGen] Solve LLVM CodeGen br instruction accept not-i1 type issue #2381 . It could solve conv2d+ PRelu crash issue on ARM CPU. We have met two threads report this issue on the forum: https://discuss.tvm.ai/t/compile-failure-after-successfull-auto-tune/1561 and https://discuss.tvm.ai/t/debug-failed-to-export-library-with-llvm-on-arm-cpu-when-using-prelu/1006/4

[ehsanmok]

@yzhliu it should be ready if the last issue with CI build resolves quickly and there's no other requested changes which seems there's none.
cc: @nhynes

[nhynes]

@yzhliu the rust frontend is almost ready. It'll probably take a few days of concerted effort to get it ready to merge, but I think it'll make it in time.

[Laurawly]

@ZihengJiang Bug fix for SSD full GPU support and SSD tutorial update #2510 could be added to the list.

Thanks to everyone who have pushed to last release cycle in the past months. We would like to propose the release of v0.5 on Jan 28th.

We encourage everyone in the community to put their weights to review and vote the release. @dmlc/tvm-team

Please reply this thread on

• Things that we missed in the release note
• Bugfixes that need to be included in this release

[ZihengJiang]

@Laurawly @FrozenGene added. Given the current status of the rust frontend, I will add it to the 0.5 release announcement. @nhynes @ehsanmok