# Welcome
## Hello There!
Welcome to the official technical documentation for the Vanna Network! Here are some quick links to help you get started:
## The Valence Network
{% content-ref url="/pages/g2N7DkwqjJimEKIoPatl" %}
[Summary](/introduction/summary)
{% endcontent-ref %}
{% content-ref url="/pages/fP82GbezvZBWMkiUYf6U" %}
[Inference Modes](/vanna-network/inference-modes)
{% endcontent-ref %}
{% content-ref url="/pages/XDAC6wD4v8Tfxo4RhfXy" %}
[Architecture](/vanna-network/architecture)
{% endcontent-ref %}
{% content-ref url="/pages/0XPXWrxqjKjxrrbcbGRu" %}
[The AI One-Stop-Shop](/vision/the-ai-one-stop-shop)
{% endcontent-ref %}
## Building on Valence
We've put together some helpful guides for you to get setup with Vanna Network quickly and easily.
{% content-ref url="/pages/lHJkOnheBjf2R06Rnwau" %}
[Getting started](/build/getting-started)
{% endcontent-ref %}
## Getting Involved
Come join us on our journey to decentralize AI inference, check our official links and communities out here!
{% content-ref url="/pages/CyJHhoaVZm8mJ1htCYy4" %}
[Links](/links)
{% endcontent-ref %}
# Summary
Verifiable AI, Everywhere.
## Vanna Network Overview
{% hint style="info" %}
Terminology Tip**:** An "inference" is just a fancy way of saying "running a trained AI/ML" model on some input parameters.
{% endhint %}
**The Vanna Network is the decentralized execution layer for AI/ML.**
Vanna supports native inference directly computed, secured, and verified on-chain; designed in a way that is so seamless that leveraging AI/ML in dApps is as easy as a simple smart contract function call.
**Vanna is the one-stop-shop for AI inference and has everything you need to run an inference pipeline on-chain.**
1. **Querying Data:** Access oracle feed data or on-chain state by making interchain queries to smart contracts.
2. **Preprocessing:** Leverage Vanna's built-in precompiles to preprocess queried raw data to prepare it for inference.
3. **Inference Execution:** Seamlessly and scalably run inference with any level of cryptographic security that suits your use-case
4. **Inference Validation:** All cryptographic proofs securing inference are verified on the Vanna network by validator nodes.
5. **Publishing & Provenance:** Inference results can be delivered to contracts on any chain through cross-chain messages, and are also published to the data availability layer.
# The AI One-Stop-Shop
Vanna is the one-stop shop for AI
**Vanna is your one-stop-shop for on-chain AI and ML inference.**
1. **Model Selection/Hosting -** Browse models on the VannaNet UI and choose a model that suits your use-case, or one-click upload your own.
2. **Data Collection** - Utilize oracle contracts on Vanna or identify smart contracts on other chains that can be accessed through interchain queries on Vanna. Composability of smart contracts allows you to use AI models like building blocks.
3. **Data Processing** - Leverage any precompiles existing on the Vanna Blockchain to preprocess the raw data collected.
4. **Inference Execution/Validation** - Execution and validation of inference is done by Vanna's infrastructure.
5. **Data Provenance** - Results can be published cross-chain and the results + any artifacts are also batch-posted to the DA layer.
Below is a more detailed graph of an end-to-end inference consumer workflow:
# AI Models on Vanna
**AI Agents**
The Vanna Network aims to be the one-stop shop for AI agents where agentic models can be deployed and will simply be able to run autonomously. This is powered not just by the suite of AI/ML-specific features Vanna offers (inference execution, preprocessing tools, access to on-chain state...etc) but also the composability of smart contracts on Vanna.
More details on our framework for AI Agents to come soon....
**AI Models**
The Vanna Labs team also plans to release traditional HTTP API endpoints and serve as a contender in the Web2.0 AI compute space. Not only is inference just as efficient as its centralized cloud compute provider counterparts, but Vanna also offers a suite of options around inference security that cloud providers do not have. Furthermore, integrating the Vanna Network as an execution layer into your AI/ML inference workloads is just as easy as other providers.
# Applied ML on Vanna
Our vision for applied ML models on the Vanna Network
In addition to AI models and agents, applied machine learning (ML) also can have a tremendous impact on the Web3.0 space by enabling existing protocols to seamlessly develop smarter features.
For example, existing problems in DeFi ranging from [AMM liquidity providers bleeding money](https://hackernoon.com/half-of-uniswap-v3-users-lose-money-heres-why) to [lack of risk management in CDP protocols](https://www.coindesk.com/markets/2021/06/17/in-token-crash-postmortem-iron-finance-says-it-suffered-cryptos-first-large-scale-bank-run/) can be made better by taking a page out of TradFi’s book and using more sophisticated models and computation.
One can imagine a world where lending protocols compute interest rates using smarter models, AMM pools reducing impermanent loss through charging dynamic fees based on predicted realized volatility, or CDP protocols having better risk engines to minimize loss during liquidation proceedings. We believe higher efficiency and effective risk management through applied ML can seriously level up DeFi’s game to increase traction and adoption. Without the same tools, it’s difficult for decentralized finance to approach traditional quantitative finance in terms of efficiency and optimization.
Applied ML in Web3.0 would allow for development of features that increases efficiency and improves risk management that can seriously level up DeFi’s game to increase traction and adoption. Without the same tools, it’s difficult for decentralized finance to approach traditional quantitative finance in terms of efficiency and optimization.
Below we've outlined a series of Web3.0 use-cases that we at Vanna Labs are particularly excited about. For more details, check out our [Medium article](https://medium.com/vanna-labs/applications-ai-ml-on-the-blockchain-dc4ef449179e) about the use-cases of ML in DeFi.
Web3.0 AI/ML Use-Cases
# Interoperability
For on-chain AI to thrive, it must have access to a wealth of data spanning various chains, decentralized applications, and oracles made available through interoperability protocols. Access to a diverse set of cross-chain data sources empowers AI models to engineer features from disparate sources, enhancing their capabilities and potential impact. For example, reputation AI models should be able to leverage historical data from multiple chains to improve their classification of whether a certain address is an authentic user or an airdrop-farming bot. Similarly, DeFi risk models should be able to leverage a wide variety of cross-chain data sources for more accurate risk assessments of the market. Interoperability unlocks a vast reservoir of data that facilitates the operation of on-chain AI.
Vanna is leveraging cross-chain messaging features for the chain. This will allow a variety of dApps across a multitude of chains to all be able to access the compute that Vanna has to offer. Traditional chains like Ethereum or Arbitrum will be able to leverage cross-chain contract calls to inference models stored in Vanna's filestore on the Vanna network.
The utility of on-chain AI would also be limited if it did not have access to programming interfaces that allowed it to operate in a multi-chain fashion. Providing access to a large number of chains and decentralized applications allows these models to perform more actions and simply have more impact.
Primarily, we foresee two types of cardinalities of on-chain AI interactions for increased scalability: one-to-many which allows the source chain to run on-chain inference and expose the result to multiple chains via a view function that can be accessed via interchain read queries, and many-to-many which allows parties on any chain to invoke cross-chain smart contract calls to run inference on the source chain in an ad-hoc fashion.
# Computational Verifiability
Importance of computational verifiability and integrity
As use of AI becomes increasingly ubiquitous, the importance of computational verifiability becomes increasingly important to ensure honest behavior. Consider the following scenario:
*"A user types in a query on ChatGPT and submits it, the query is sent as an API request to OpenAI's servers where OpenAI inferences their GPT model with the query, then returns the result back to the client (user) where the result is displayed on the site"*
OpenAI claims that they run the query through GPT4, but the user has no idea if they used the correct query, or more importantly, *the correct model*. The user is completely trusting the company to do the right thing. This is dangerous becuase running inference on large AI models is extremely expensive, so it's completely possible that companies could bait-and-switch and choose to use cheaper/smaller models instead to reduce their costs. After all, who would know?
Ensuring comptuational verifiability with ZKML
That's why Vanna Labs is developing cryptographic schemes as enshrined primitives on the network that gives inference the consumers the option to have their inference requests cryptographically secured. For more information about Vanna's security models around inference, check out:
{% content-ref url="/pages/fP82GbezvZBWMkiUYf6U" %}
[Inference Modes](/vanna-network/inference-modes)
{% endcontent-ref %}
In addition to cryptographic proofs that are validated on the Vanna Network, the inference results and artifacts will all be posted to a data availability layer where they can be inspected or verified by anyone.
**Computational verifiability becomes increasingly important as the implications of the inference result increases.**
What if a fintech company uses cheaper models which results in inaccurate risk assessments? What if a healthcare AI company uses a cheaper model which results in incorrect diagnoses? The Vanna Network and its enshrinement of zkML as a primitive helps to guarantee computational verifiability to protect against such scenarios.
# Decentralization
Decentralizing and open-sourcing model development
As opposed to the status quo of closed-source model development, Vanna Labs believes in open-sourcing models through its decentralized platform. When researchers and developers share their models, algorithms, and findings with the global community, it fosters collaboration, innovation, and the rapid exchange of ideas.
Vanna's models will be stored in IPFS, a P2P decentralized storage network. Any participant is able to run an IPFS storage node and access the models through the model's public CID (content identifier).
*Note: In the future, Vanna will develop technical solutions that allow for private model storage.*
The open-source approach allows a diverse range of contributors, often from different backgrounds and organizations, to collectively improve upon existing models. This collaborative effort not only accelerates the development of AI technologies but also enables the community to address challenges and iterate on solutions more efficiently.
Moreover, open-sourcing promotes transparency, as the inner workings of models become accessible to other developers. This transparency not only facilitates trust and understanding among users but also encourages scrutiny and constructive feedback from the broader community. The iterative feedback loop created through open-source development can lead to the identification and resolution of issues more swiftly, ultimately resulting in more robust and reliable AI models.
Additionally, the democratization of AI knowledge through open-source initiatives helps democratize access to cutting-edge technologies, leveling the playing field for researchers and practitioners around the world, and contributing to the democratization of artificial intelligence itself. In summary, the open-sourcing of AI model development and research acts as a catalyst for collective advancement, fostering innovation, transparency, and a faster pace of development.
# Censorship Resistance
Importance of fair distribution of AI technology
Censorship-resistance is of paramount importance for AI models as it ensures the preservation of free expression, diversity of thought, and protection against unwarranted restrictions on information dissemination. However, the structural centralization of AI makes it prone to censorship and also creates a single point of failure.
Having models hosted on centralized compute platforms like AWS also creates a troubling single point of failure, if these platforms were to go down it would create an outright outage of access to the models.
Additionally, model providers can also choose to censor traffic from certain parties or regions, furthering reducing access. In such a scenario, the undue influence or control of that single provider could lead to widespread censorship or blatant manipulation of AI-generated content.
Censorship-Resistance on the Vanna Network
The Vanna Labs team believes in creating an open-platform where any party can decentralize storage of their models, and also permissionlessly allow for model inference so both developers and users can always have access to AI.
By promoting censorship-resistance, AI models can contribute to a more open and democratic information landscape, fostering innovation and preventing the undue influence of a single authority over the flow of knowledge. Censorship-resistance, therefore, not only safeguards against deliberate censorship but also mitigates the risk of unintentional disruptions. A decentralized approach to AI hosting promotes resilience, transparency, and a more robust defense against the concentration of power that could compromise the integrity of AI systems.
# Architecture
High-level Vanna Network Architecture
Vanna is an EVM-compatible blockchain that can support native AI/ML inference directly computed, secured, and verified on-chain. The network will store both generic block data and inference transaction data onto a DA layer.
The Vanna Network also has its own decentralized filestore layer built off of an IPFS swarm. The CID hash from model storage is used as the model identifier on the Vanna Network for inference.
For a better understanding of the data flow of on-chain inference and how it works on the Vanna Network, take a look at the data flow diagram below.
For more details on our vision for an end-to-end inference process on-chain, check out the AI One-Stop-Shop section.
{% content-ref url="/pages/0XPXWrxqjKjxrrbcbGRu" %}
[The AI One-Stop-Shop](/vision/the-ai-one-stop-shop)
{% endcontent-ref %}
# Data Preprocessing
The Vanna blockchain will contain a variety of precompile functions that help preprocess raw data in preparation for running inference on AI or ML models.
Currently in development...
# Parallelized Inference Pre-Execution (PIPE)
In order to achieve maximum throughput and performance on the Vanna Network, we designed a novel EVM execution framework called Parallelized Inference Pre-Execution (PIPE). PIPE allows Vanna to execute transactions and inferences in the most efficient way, providing a seamless interaction between model inference and the EVM.
One of Vanna's main goals is supporting AI inference on the blockchain in a highly scalable way. Complex models however can be computationally expensive, which poses a risk when embedding them into native blockchain transactions. Slow model inferences could severely impact the overall performance of the network, slowing it down for all users.
Instead, Vanna designed a 3-phase approach when executing inferences embedded into transactions.
### First-Phase: Simulation
In this first phase, Vanna runs every transaction through a simulator in order to find out what inference requests the transaction will make. In normal execution, when a transaction makes an inference request, the network would have to execute the inference synchronously, while blocking the EVM and all other transactions as well. In our simulation however, we only record the inference request(s) sent by each transaction, we don't actually execute them.
Once the simulation is done, the transaction is returned to the Vanna Node, along with all the inference requests that were made.
### Second-Phase: Inference Mempool
In the second phase, transactions and their inference requests are added to a special Mempool that sends off the requests to the Vanna Inference Nodes. Transactions are kept in the mempool while their corresponding requests are being executed. Once all requests made by a particular transaction are done, it can be submitted to be executed in the EVM.
### Third-Phase: EVM Execution
Finally, we inject the inference results into the EVM so the transaction can read it directly, just like it would read any other variable. The transaction is then executed and committed to the blockchain. Because no actual inference takes place at this stage, execution is blazing fast, and has no impact on the overall performance of the network.
High-level overview of our execution framework.
### Benefits
The benefits of this approach are:
* No single transaction can slow down other transactions, or even worse the entire blockchain
* Transactions that use faster models can be executed as soon as inference is completed
* Transactions that do not use model inference at all are guaranteed to execute immediately
* Network resources are used in the most efficient way
* The network's performance can be scaled horizontally by running more AI inference nodes
* No modifications required to smart contracts, SIF is implemented entirely at the execution layer
Alternative execution models are characterized by slow and unreliable performance, and open up a new vector of DDoS attacks on the network.
To sum up, SIF is one of the core technologies that allows Vanna to deliver the seamless and performant AI inference on the blockchain.
# Validation-Computation Separation (VCS)
The Vanna Network employs two types of nodes, validator nodes and inference nodes, and bifurcates network validation and inference computation to the two types of nodes respectively.
**Stateful Validation** - The rollup nodes independently validate transactions and verify the state of the Vanna Network. Rollup nodes on the Vanna Network also participate in validating cryptographic proofs generated from inference nodes.
**Stateless Computation** - The inference nodes do not validate transactions and blocks on the network, and solely focus on computing AI/ML inference as well as generating cryptographic proofs for the inference.
This unique design allows the Vanna Network to scale without making compromises to decentralization. Hardware requirements continue to remain low for stateful validator nodes while the inference nodes do the heavy lifting (inference), with the validator nodes participating in proof validation to enforce honest computation by inference nodes.
# Cryptoeconomic Security
The Vanna Network offers cryptoeconomic security in the form of a staking contract on the application layer. When inference nodes come online to participate in securing the network, they must post Vanna tokens as collateral in the staking contract. The staking contract enforces behavior of the inference nodes, with slashing conditions including but not limited to:
1. **zkML** - Generating proofs that are invalid and cannot be verified cryptographically
2. **opML** - Successful challenges to the inferences generated by the node
3. **zkFP** - Successful challenges to the inferences generated by the node, or the inability to generate a ZK SNARK proving the inference
# Modular Design
# Inference Modes
Types of Inference on the Vanna Network
Vanna supports a variety of flavors of inference to suit all sorts of use-cases depending on demands for speed, cost, verifiability, and security.
* **Vanilla Inference**
* Fast execution of inference through leveraging GPU hardware acceleration and transaction parallelization
* **zkML Inference**
* Inference where the inference node also subsequently generates a zkML proof with EZKL. The proof is then validated by validator nodes on the Vanna Network for an absolute guarantee of security.
* **zkFP Inference**
* Inference where the inference computation is assumed to be honest, but makes the transaction available for challenge. If challenged, the challenger puts up a stake and the inference node must provide a zkML proof proving the result.
* **opML Inference**
* Inference where the inference computation is assumed to be honest, but makes the transaction available for challenge. If challenged, the challenger puts up a stake and a bisection verification game is played to see whether the inference node was honest.
* **TEE Inference**
* Run a private enclave node with the model stored locally, so the model weights remain private. Inference requests for the model are routed to that node, with a zkML proof subsequently generated and verified by the Vanna Network.
# Vanilla Inference
Vanilla inference represents a regular inference to an AI or ML model in Vanna's decentralized filestore.
In order to run basic vanilla inference, you need to implement the ArbInference interface for inferCall.
**model** - The IPFS CID for the model
**input** - The input parameters to the inference request
{% code overflow="wrap" %}
```solidity
function inferCall(bytes memory model, bytes memory input) external returns (bytes memory);
```
{% endcode %}
Below is an example of a minimal smart contract that runs vanilla inference:
{% code title="ArbInference.sol" %}
```solidity
pragma solidity >=0.4.21 <0.9.0;
/// @title Infer Call
/// @notice This contract is used to run on-chain inference.
/// This custom contract will set on 0x000000000000000000000000000000000000011a since we set it in precompile.go.
interface ArbInference {
function inferCall(bytes memory model, bytes memory input) external returns (bytes memory);
function inferCallZK(bytes memory model, bytes memory input) external returns (bytes memory);
}
```
{% endcode %}
{% code title="VanillaInference.sol" %}
```solidity
pragma solidity ^0.8.18;
import "contracts/ArbInference.sol";
contract VanillaInference {
string value;
function infer(
string calldata modelName,
string calldata inputData
) public {
value = string(abi.encodePacked(ArbInference(address(0x11a)).inferCall(abi.encodePacked(modelName), abi.encodePacked(inputData))));
}
function getValue() public view returns (string memory) {
return value;
}
}
```
{% endcode %}
# zkML Inference
zkML inference represents running inference on an AI or ML model in Vanna's decentralized filestore, where a [ZK-SNARK](https://z.cash/learn/what-are-zk-snarks/) will also be generated and validated by the nodes on the Vanna Network. The proof will be available on the data availability layer, which can be queried through our inference transaction explorer.
In order to run zkML inference, you need to implement the ArbInference interface for inferCallZK.
**model** - The IPFS CID for the model
**input** - The input parameters to the inference request
{% code overflow="wrap" %}
```solidity
function inferCallZK(bytes memory model, bytes memory input) external returns (bytes memory);
```
{% endcode %}
Below is an example of a minimal smart contract that runs zkML inference:
{% code title="ArbInference.sol" %}
```solidity
pragma solidity >=0.4.21 <0.9.0;
/// @title Infer Call
/// @notice This contract is used to run on-chain inference.
/// This custom contract will set on 0x000000000000000000000000000000000000011a since we set it in precompile.go.
interface ArbInference {
function inferCall(bytes memory model, bytes memory input) external returns (bytes memory);
function inferCallZK(bytes memory model, bytes memory input) external returns (bytes memory);
}
```
{% endcode %}
{% code title="zkMLInference.sol" %}
```solidity
pragma solidity ^0.8.18;
import "contracts/ArbInference.sol";
contract zkMLInference {
string value;
function zkInfer(
string calldata modelName,
string calldata inputData
) public {
value = string(abi.encodePacked(ArbInference(address(0x11a)).inferCallZK(abi.encodePacked(modelName), abi.encodePacked(inputData))));
}
function getValue() public view returns (string memory) {
return value;
}
}
```
{% endcode %}
# zkFP Inference
zkFP inference represents running inference on an AI or ML model in Vanna's decentralized filestore in an optimistic fashion with no proof directly generated to secure inference. However, the staking contract allows for the inference consumer to post a stake and submit a challenge to the inference within a challenge period. If challenged, the node that ran the inference generates a [ZK-SNARK](https://z.cash/learn/what-are-zk-snarks/) that acts as a fraud proof and will be validated by the nodes on the Vanna Network. On successful challenges, the inference node is slashed; on unsuccessful challenges, the challenger's stake is lost. The proof will be available on the data availability layer, which can be queried through our inference transaction explorer.
In order to run zkFP inference, you need to implement the ArbInference interface for inferCallZK.
**model** - The IPFS CID for the model
**input** - The input parameters to the inference request
{% code overflow="wrap" %}
```solidity
function inferCallZKFP(bytes memory model, bytes memory input) external returns (bytes memory);
```
{% endcode %}
Below is an example of a minimal smart contract that runs zkFP inference:
{% code title="ArbInference.sol" %}
```solidity
pragma solidity >=0.4.21 <0.9.0;
/// @title Infer Call
/// @notice This contract is used to run on-chain inference.
/// This custom contract will set on 0x000000000000000000000000000000000000011a since we set it in precompile.go.
interface ArbInference {
function inferCall(bytes memory model, bytes memory input) external returns (bytes memory);
function inferCallZK(bytes memory model, bytes memory input) external returns (bytes memory);
function inferCallZKFP(bytes memory model, bytes memory input) external returns (bytes memory);
}
```
{% endcode %}
{% code title="zkFPInference.sol" %}
```solidity
pragma solidity ^0.8.18;
import "contracts/ArbInference.sol";
contract zkFPInference {
string value;
function zkfpInfer(
string calldata modelName,
string calldata inputData
) public {
value = string(abi.encodePacked(ArbInference(address(0x11a)).inferCallZKFP(abi.encodePacked(modelName), abi.encodePacked(inputData))));
}
function getValue() public view returns (string memory) {
return value;
}
}
```
{% endcode %}
# opML Inference
# TEE Inference
# Data Storage
# Model Storage
The AI/ML models will be stored on Vanna's swarm on IPFS, for accessing the IPFS swarm and connecting to it see the link below.
{% content-ref url="/pages/26v5MvwQ1pcNFcoBawT4" %}
[Model Storage](/build/getting-started/model-storage)
{% endcontent-ref %}
Storing AI/ML models on a permissionless storage network like IPFS leverages decentralization, mitigating the risk of single points of failure and ensuring model availability even in the absence of centralized servers. This resilience enhances reliability and robustness, a crucial factor for ensuring inference liveness on the Vanna Network.
Moreover, IPFS fosters transparency by enabling easy tracking of model history and versioning, facilitating collaborative development and verification processes. This transparency promotes trust among stakeholders and encourages participation in model improvement and validation efforts. Overall, utilizing IPFS for model storage empowers a more resilient, transparent, and collaborative AI/ML ecosystem that fits Vanna's ethos.
# Data Availability
### Data Availability
Vanna will be leveraging data availability (DA) layers that provides an effective solution to the [data availability problem](https://docs.celestia.org/learn/how-celestia-works/data-availability-layer).
# Portal
An Open Platform
Vanna Labs is developing an open platform for hosting and sharing models (similar to HuggingFace) where users can share both models and datasets, and help iterate on each other's approaches to applications of AI/ML to web3.
VannaNet will serve as a user-friendly interface to Vanna's decentralized filestore of models.
Currently in development...
# Getting started
Getting set up on the Valence network
Here you will be able to find the tools you need to develop dApps and models on the Valence network.
**List of deployed networks:**
{% content-ref url="/pages/M6SxqzQ8cL84SNhL2gRb" %}
[Networks](/build/getting-started/networks)
{% endcontent-ref %}
**Faucets:**
{% content-ref url="/pages/2ADcW9gxa8kCIsFNVEsO" %}
[Faucets](/build/getting-started/faucets)
{% endcontent-ref %}
**Model Storage:**
{% content-ref url="/pages/26v5MvwQ1pcNFcoBawT4" %}
[Model Storage](/build/getting-started/model-storage)
{% endcontent-ref %}
# Networks
Connecting to the Valence Blockchain
## Valence Network Testnets
1. **Pareto** - Testnet that is an official replica of mainnet, gearing up for mainnet launch
2. **Euler** - Ongoing experimental testnet for testing new features
3. **Mainnet** - Final Launch of Vanna Blockchain Mainnet on Ethereum
## Vanna Network Endpoints
# Model Storage
Valence Network's Model Storage Network on IPFS
In order to permissionlessly access open-source models on Valence's decentralized model storage layer, set up an IPFS client and connect to Vanna's swarm with the following swarm key:
```
/key/swarm/psk/1.0.0/
/base16/
4b0c480e8a225ceaebf7eef483cb45f493bf45f2e7eb65442d6e716c136846a1
```
To learn more about IPFS and how to run an IPFS node, refer to the [IPFS documentation](https://docs.ipfs.tech/).
# Building dApps
App development on Valence
Valence is an EVM chain that's compatible with most existing frameworks and tools out there. In addition to the standard EVM capabilities, we also support native AI inference, directly from your smart contract.
Inference capabilities on Valence are exposed through a number of Solidity interfaces and precompiles that any smart contract can call.
Our Solidity Inference SDK can be installed by running:
`npm i valence-inference-lib`
The SDK is split into 2 parts:
* **Data preprocessing**: helps convert on-chain data into a format that's suitable for inferencing
* **Inference**: used for natively running inference from smart contracts
# Data Preprocessing
TODO
# Inference API
Describes the interfaces Valence exposes for using inference in smart contracts.
{% hint style="info" %}
Use `npm i valence-inference-lib` to install our Solidity library.
{% endhint %}
### **Inference Precompile**
Valence Inference is provided through a standard Interface that any smart contract can use. The inference is implemented by a custom precompile called `IValenceInference`.
{% hint style="info" %}
The Inference precompile and its functions are accessible at address `0x00000000000000000000000000000000000000F4`
{% endhint %}
Valence exposes various types of inference, ranging from LLMs to classical ML models. In addition, it supports various security techniques, such as ZKML and TEE inference. Developers have the option to choose the most suitable methods for their use-case and requirements.
The 2 high-level functions exposed by the inference precompile are `runModel` and `runLllm`. `runLllm` is a function specifically designed for running large language models, whereas `runModel` is a generic method that can be used to execute any type of AI and ML models.
```solidity
interface IValenceInference {
enum ModelInferenceMode { VANILLA, ZK }
enum LlmInferenceMode { VANILLA, TEE }
function runModel(
ModelInferenceMode mode,
ModelInferenceRequest memory request
) external returns (ModelOutput memory);
function runLlm(
LlmInferenceMode mode,
LlmInferenceRequest memory request
) external returns (LlmResponse memory);
}
```
### LLM Inference Requests
The request and response for LLMs (`runLlm`) are defined as follows. The main input is a prompt, and the answer is returned as a string as well. In addition, there are some common LLM parameters that can be tuned.
The list of LLMs you can use can be found in [Supported LLMs](/build/models/supported-llms).
struct LlmInferenceRequest {
string model; // ID of the LLM to use
string prompt; // LLM prompt
uint32 max_tokens; // max tokens to generate
string[] stop_sequence; // stop sequences for model response
uint32 temperature; // model temperature (between 0 and 100)
}
struct LlmResponse {
string answer; // answer generated by the LLM
bool is_simulation_result; // indicates whether the result is real
}is_simulation_resu
To read more about `is_simulation_result`, please see [#simulation-results](#simulation-results "mention")
### Generic Inference Requests
For all other ML models, the input and response can take various shapes and forms (such as numbers and strings), so we built a flexible framework that lets you define any type of input that your ONNX model expects. The input is made up of an array of number tensors and string tensors. You only need to set the tensors that your model expects (eg can leave string tensors empty if your model only expects numbers).
```solidity
/**
* Can be used to represent a floating-point number or integer.
*
* eg 10 can be represented as Number(10, 0),
* and 1.5 can be represented as Number(15, 1)
*/
struct Number {
int128 value;
int128 decimals;
}
/**
* Represents a model tensor input filled with numbers.
*/
struct NumberTensor {
string name;
Number[] values;
}
/**
* Represents a model tensor input filled with strings.
*/
struct StringTensor {
string name;
string[] values;
}
/**
* Model input, made up of various tensors of numbers and/or strings.
*/
struct ModelInput {
NumberTensor[] numbers;
StringTensor[] strings;
}
/**
* Model inference request.
*/
struct ModelInferenceRequest {
string modelId;
ModelInput input;
}
/**
* Model output, made up of tensors of either numbers or strings, ordered
* as defined by the model.
*
* For example, if a model's output is: [number_tensor_1, string_tensor_1, number_tensor_2],
* you could access them like this:
*
* number_tensor_1 = output.numbers[0];
* string_tensor_1 = output.strings[0];
* number_tensor_2 = output.numbers[1];
*
*/
struct ModelOutput {
NumberTensor[] numbers;
StringTensor[] strings;
bool is_simulation_result; // indicates whether the result is real
}
```
To read more about `is_simulation_result`, please see [#simulation-results](#simulation-results "mention")
### Simulation Results
Both `LlmResponse` and `ModelOutput` have a flag called `is_simulation_result` that indicates whether the result returned is "real" or not. As explained in [Parallelized Inference Pre-Execution (PIPE)](/vanna-network/architecture/parallelized-inference-pre-execution-pipe), Vanna transactions are executed in 2 phases. In the first phase, the transaction is executed in simulation mode to gather and execute all inference requests in the background. Once the results are ready, the transaction is re-executed with the actual inference results. `is_simulation_result` indicates whether the transaction is currently being executed in simulation mode or not. When it it set to `false`, the value returned is coming from the model, however, when it is set to `true`, the value is empty and developers should explicitly handle this scenario in their code.
{% hint style="info" %}
Transaction simulation results are never committed to the blockchain
{% endhint %}
For example:
```solidity
function calculateFeeFromModelResult(ModelResult memory result) int128 {
if (result.is_simulation_result) {
// when in simulation, return some sensible default value
return 1;
}
return result.numbers[0].values[0].value * 2;
}
```
# Inference Example
### **Example smart contract**
Below is a smart contract that utilizes the `IVannaInference` interface to run inference on-chain.
```solidity
import "valence-inference-lib/src/IInference.sol";
contract InferenceExample {
// Execute an ML model from Valence's storage layer, secured by ZKML
function runZkmlModel() public {
ModelInput memory modelInput = ModelInput(
new NumberTensor[](1),
new StringTensor[](0));
Number[] memory numbers = new Number[](2);
numbers[0] = Number(7286679744720459, 17); // 0.07286679744720459
numbers[1] = Number(4486280083656311, 16); // 0.4486280083656311
modelInput.numbers[0] = NumberTensor("input", numbers);
ModelOutput memory output = INFERENCE_CONTRACT.runModel(
IInference.ModelInferenceMode.ZK,
ModelInferenceRequest(
"QmbbzDwqSxZSgkz1EbsNHp2mb67rYeUYHYWJ4wECE24S7A",
modelInput
));
if (output.is_simulation_result == false) {
resultNumber = output.numbers[0].values[0];
} else {
resultNumber = Number(0, 0);
}
}
// Execute a Large Language Model directly in your smart contract
function runLlm() public {
string[] memory stopSequence = new string[](1);
stopSequence[0] = "";
LlmResponse memory llmResult = INFERENCE_CONTRACT.runLlm(
IInference.LlmInferenceMode.VANILLA,
LlmInferenceRequest(
"meta-llama/Meta-Llama-3-8B-Instruct",
"Hello sir, who are you?\n",
1000,
stopSequence,
0
));
if (llmResuklt.is_simulation_result) {
resultString = "empty";
} else {
resultString = llmResult.answer;
}
}
}
```
# Models
# Supported LLMs
Valence supports a number of capable and versatile LLMs out-of-the box. You can use any of these models with the `runLlm` function defined in [Inference API](/build/building-dapps/inference-api)
You can find the list below:
Model
Model ID
Llama-3-8B-Instruct
meta-llama/Meta-Llama-3-8B-Instruct
# Links
Official Vanna Labs Links
## Social Media
Website:
Twitter:
Discord:
LinkedIn:
## Writing
Medium Publication:
## Technical
Github:
# AI x Blockchain
Challenges arising from the Web3.0 x Blockchain integration
## Integration Challenges
Directly integrating inference directly into modern blockchains is challenging and infeasible due to infrastructure limitations. See our [medium article](https://medium.com/vanna-labs/the-state-of-blockchain-x-ai-inference-c0e386356d49) for more context.
The AI x Blockchain Trilemma
The same infamous trilemma that plagues the blockchain ecosystem can be similarly applied to the problem of taking AI on-chain.
**Decentralization (AI Computational Limitations):** Model inference can be computationally expensive, on-chain inferences at scale with modern blockchain architecture would be infeasible without steeply raising the computational requirements to run a full node, which would be a detriment to decentralization.
**Security (AI Inference Integrity):** As AI models can many times be a black-box, heavy reliance on these inferences could expose protocols to new vectors of attack. Relying on oracles to bring model inference results on-chain have no guarantees in the integrity of the inference, and allow for potential exploits.
**Scalability (Replication of state through AI Inference):** Model inference can potentially be computationally expensive, in traditional proof-of-stake consensus full nodes re-execute transactions for block validation via Merkle proofs so even if native inferencing is implemented each inference would be executed hundreds of thousands of times which is computationally inefficient and could severely congest the network.
## **The Vanna Labs Solution**
**Vanna Labs is creating a scalable and secure AI execution layer that resolves the challenges above.**
**Scalability -** The Vanna Network is able to scale its network through powerful inference nodes that featuring hardware acceleration that is designed to run inference.
{% content-ref url="/pages/XDAC6wD4v8Tfxo4RhfXy" %}
[Architecture](/vanna-network/architecture)
{% endcontent-ref %}
**Decentralization -** The Vanna Network achieves decentralization through Validation-Computation Separation (VPS). The network remains decentralized since hardware requirements for validator nodes remain low since the computationally-intensive inference is done by inference nodes, the validator nodes just focus on validating proofs generated by inference nodes.
{% content-ref url="/pages/LrdJetAfQJf1xOm8EmBg" %}
[Validation-Computation Separation (VCS)](/vanna-network/architecture/validation-computation-separation-vcs)
{% endcontent-ref %}
**Security -** The Vanna Network achieves secure computation through a variety of cryptographic schemes that validate the computation done by inference nodes. For more information on these inference modes, see:
{% content-ref url="/pages/fP82GbezvZBWMkiUYf6U" %}
[Inference Modes](/vanna-network/inference-modes)
{% endcontent-ref %}
# Optimistic Rollups
Scalability thr
## What is an Optimistic Rollup?
An optimistic rollup is scaling solution that allows for off-chain transactions while still inheriting the security of the underlying blockchain.
Optimistic rollups work by moving computation and state storage off-chain. This means that transactions are processed and stored off of the underlying blockchain, which frees it up to handle more transactions.
When a transaction is processed off-chain, it is bundled together with other transactions into a batch. This batch is then submitted to the underlying blockchain, along with a fraud proof. The fraud proof is a mathematical proof that the batch of transactions was processed correctly.
If anyone believes that a batch of transactions was processed incorrectly, they can challenge the fraud proof. If the challenge is successful, the batch of transactions is rolled back and the correct state is restored.
Optimistic rollups offer a number of advantages over other blockchain scaling solutions. They are relatively simple to implement, they are secure, and they are compatible with the Ethereum ecosystem.
Here are some advantages of the Optimistic Rollup scaling solution:
* **Increased scalability:** Optimistic rollups can process thousands of transactions per second, which is much faster than Ethereum mainnet.
* **Reduced fees:** Optimistic rollups can also reduce transaction fees, making it more affordable to use Ethereum.
* **Security: T**he fraud proof mechanism ensures that transactions are processed correctly.
* **Compatibility:** Optimistic rollups are often EVM-compatible, making it compatible with the Ethereum ecosystem. This means that you can use your existing Ethereum wallets and dApps with optimistic rollups.
# Data Availability
Data Provenance for Blockchains
In blockchains, data availability refers to the ability of nodes to download the data contained within all blocks propagated through a peer-to-peer network. This is important because it ensures that all nodes have access to the same information, which is necessary for verifying the integrity of the blockchain.
There are a number of factors that can affect data availability in blockchains. One factor is the size of the blockchain. As the blockchain grows, it becomes more difficult for nodes to download and store all of the data. Another factor is the network bandwidth. If the network bandwidth is limited, it can slow down the process of downloading and storing blockchain data.
Vanna will be using a general data-availability layer to ensure that blockchain transaction data as well as any inference artifacts are stored and retrievable. Vanna will also be using Arweave to store key objects used in zkML proof generation; notably, the structured reference strings (SRS) for proof-generation and proof-validation. This ensures that the SRS needed to validate proofs by verifiers are permanently and universally available. To read more about Arweave, check them out [here](https://www.arweave.org/).
# zkML
Securing AI/ML Inference through ZK Technology
**Zero Knowledge Machine Learning (zkML)**
The Vanna Network incorporates zkML technology to ensure that the inferences computed by the network are honest and correct. But what exactly is zkML?
A zero-knowledge (ZK) proof is a cryptographic protocol in which one party, the prover, can prove to another party, the verifier, that a given statement is true, without revealing any additional information beyond the fact that the statement is true.
Similarly, zero-knowledge machine learning (zkML) uses zero-knowledge proofs (ZKPs) to allow a party (the prover) to demonstrate to another party (the verifier) that a certain statement is true, without revealing any additional information. In the context of machine learning, this could mean proving that a model correctly classifies a certain input or runs an inference correctly.
Vanna Labs will be using [EKZL.xyz](https://ezkl.xyz/) for generating and validating zkML proofs on its network. EZKL is an [open-sourced library](https://github.com/zkonduit/ezkl) that uses the Halo2 proof system. Using EZKL, the Vanna Network can create ZK proofs that cryptographically prove statements like:
“*I ran this publicly available neural network on some private data and it produced this output*”
or
“*I ran my private neural network on some public data and it produced this output*”
These cryptographic proofs allows Vanna to guarantee the integrity of AI/ML inference computations, and prevents nodes from computing bad inferences that could pose as security risks in the blockchain network.
So… how does this magic work? We’d highly recommend looking through [ezKL’s blog](https://blog.ezkl.xyz/) as well as their [docs](https://docs.ezkl.xyz/) to get a stronger technical understanding of zkML, but here’s a high-level overview.
1. **Compilation:** The AI model is converted into a mathematical circuit using tools like [Circom](https://docs.circom.io/). In the context of ZK proofs, tools like Circom are commonly used to describe computations as circuits in a high-level domain-specific language.
2. **Setup Phase:** A trusted party generates public parameters that will be used in the proof system, e.g. using a multi-party compute (MPC) like the [Perpetual Powers of Tau](https://github.com/privacy-scaling-explorations/perpetualpowersoftau). The proving keys and verifying keys for the system are also generated.
3. **Witness Generation:** The actual data (witness) related to the statement is mapped into the circuit.
4. **Proof Generation:** Using the proving key and the mapped witness, a prover generates a succinct proof.
5. **Proof Validation:** The generated proof is made available along with the statement to verifiers. The verifiers can use the public verification key to validate a proof cheaply in constant time.
An important key factor to keep in mind is that proof generation is quite computationally expensive, but proof validation is very cheap. Because of this, it makes sense for projects like Vanna to adopt an architecture where a proof is generated once, but validated numerous times by many nodes across the network.
# opML
Securing AI/ML Inference through Optimistic Fraud Proof Technology
**Optimistic Machine Learning (opML)**
Optimistic machine learning is another solution to decentralizing AI inference, with “optimistic” referring to the idea of optimistically trusting the results of an inference unless (or until) someone challenges the result.
In the event of a challenge, the challenger first puts up a financial stake and a verification game is played, where a bisection protocol is used to locate the disputed step that caused the divergence in computation. The arbitrator contract resolves the challenge, heavily slashing the inference computer or the challenger depending on the success or failure of the challenge respectively.
This optimistic security model comes with both pros and cons compared to zkML. The biggest advantage is the lower time/computational cost that comes with adopting a model that doesn’t generate a proof on every inference, which is significantly lower especially with larger models where both the inference and proof generation can be extremely expensive. However, the biggest disadvantage is the lack of the immediate cryptographic security that zkML offers that high-stakes use-cases may require.
