🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization, pruning, distillation, etc. It compresses deep learning models for downstream d…

Standalone Flash Attention v2 kernel without libtorch dependency

Extension for Automatic1111's Stable Diffusion WebUI, using OnnxRuntime CUDA execution provider to deliver high performance result on Nvidia GPU.

2-5X faster 70% less memory QLoRA & LoRA finetuning

optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052

Google Drive Download Python Script

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

A tool to modify ONNX models in a visualization fashion, based on Netron and Flask.

🤗 Diffusers: experiment of diffusion ONNX models

Let us democratise high-resolution generation! (arXiv 2023)

ONNX Runtime support for SAM

NVIDIA® TensorRT™, an SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applicat…

🤗 Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch.

Open Neural Network Exchange

Reference implementations of inference benchmarks

Tutorials for creating and using ONNX models

TensorFlow code and pre-trained models for BERT

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit