Lux Consensus
SDK

Documentation

Python SDK

The Python SDK provides Cython bindings to the high-performance C consensus library, enabling Python applications to leverage Lux Consensus with near-native performance.

Installation

Prerequisites

# Install build dependencies
pip install cython setuptools wheel

# Install the Lux C library first
cd pkg/c && make && make install

Install Python Package

cd pkg/python
python setup.py install

Quick Start

import lux_consensus as lux

# Create consensus configuration
config = lux.ConsensusConfig(
    k=20,                           # Consecutive successes
    alpha_preference=15,           # Preference quorum
    alpha_confidence=15,           # Confidence quorum
    beta=20,                       # Confidence threshold
    engine_type=lux.EngineType.DAG  # DAG consensus
)

# Create consensus engine
engine = lux.ConsensusEngine(config)

# Create and add a block
block_id = b'\x01' * 32
parent_id = b'\x00' * 32

block = lux.Block(
    block_id=block_id,
    parent_id=parent_id,
    height=1,
    data=b'transaction data'
)

engine.add_block(block)

# Process votes
voter_id = b'\x02' * 32
vote = lux.Vote(
    voter_id=voter_id,
    block_id=block_id,
    is_preference=False
)

engine.process_vote(vote)

# Check block status
if engine.is_accepted(block_id):
    print("Block accepted!")

# Get consensus statistics
stats = engine.get_stats()
print(f"Blocks accepted: {stats.blocks_accepted}")
print(f"Votes processed: {stats.votes_processed}")
print(f"Avg decision time: {stats.average_decision_time_ms:.2f}ms")

MLX GPU Acceleration

Enable Apple Silicon GPU acceleration for high-throughput consensus:

Installation with MLX

# Install with MLX support (Python 3.11 required)
pip3.11 install lux-consensus[mlx]

# Or install MLX separately
pip3.11 install lux-consensus mlx

Usage

from lux_consensus.mlx_backend import MLXConsensusBackend

# Create GPU-accelerated backend
backend = MLXConsensusBackend(
    device_type="gpu",           # "gpu" or "cpu"
    batch_size=32,               # Optimal batch size
    enable_quantization=True,    # Use int8 quantization
    cache_size=5000              # Blocks to cache on GPU
)

print(f"Device: {backend.get_device_name()}")
print(f"GPU enabled: {backend.gpu_enabled}")

# Process votes on GPU
votes = [
    (voter_id, block_id, False)
    for voter_id, block_id in vote_pairs
]

processed = backend.process_votes_batch(votes)
print(f"Processed {processed} votes on GPU")

# Adaptive batch processing
from lux_consensus.mlx_backend import AdaptiveMLXBatchProcessor

processor = AdaptiveMLXBatchProcessor(backend)

for voter_id, block_id in vote_stream:
    processor.add_vote(voter_id, block_id, False)

processor.flush()
print(f"Throughput: {processor.get_throughput():.0f} votes/sec")

Performance

Tested on M1 Max (10-core CPU, 32-core GPU):

  • Single vote: 850 ns (GPU overhead)
  • Batch 100: 6.25x speedup (8 μs GPU vs 50 μs CPU)
  • Batch 1K: 13.7x speedup (35 μs GPU vs 480 μs CPU)
  • Batch 10K: 25-30x speedup (140-190 μs GPU vs 4.2-4.8 ms CPU)

Memory Usage:

  • CPU mode: ~100 MB for 10K blocks
  • MLX GPU mode: ~250 MB (includes GPU buffers)
  • Peak: ~400 MB during large batch processing

Engine Types

class EngineType:
    CHAIN = 0   # Linear blockchain consensus
    DAG = 1     # DAG-based consensus (default)
    PQ = 2      # Post-quantum consensus

API Reference

ConsensusConfig

Configuration for the consensus engine:

config = lux.ConsensusConfig(
    k=20,                              # Consecutive successes for finality
    alpha_preference=15,              # Quorum size for preference
    alpha_confidence=15,              # Quorum size for confidence
    beta=20,                          # Decision threshold
    concurrent_polls=1,               # Max concurrent polls
    optimal_processing=1,             # Optimal processing flag
    max_outstanding_items=1024,       # Max outstanding items
    max_item_processing_time_ns=2_000_000_000,  # Timeout in nanoseconds
    engine_type=lux.EngineType.DAG   # Engine type
)

Block

Represents a block in the consensus:

block = lux.Block(
    block_id=bytes(32),    # 32-byte block identifier
    parent_id=bytes(32),   # 32-byte parent block ID
    height=int,            # Block height
    timestamp=int,         # Unix timestamp (optional)
    data=bytes             # Block data (optional)
)

# Properties
block.id          # Block ID (bytes)
block.parent_id   # Parent block ID (bytes)
block.height      # Block height (int)
block.timestamp   # Block timestamp (int)

Vote

Represents a vote on a block:

vote = lux.Vote(
    voter_id=bytes(32),    # 32-byte voter identifier
    block_id=bytes(32),    # 32-byte block ID being voted on
    is_preference=bool     # True for preference vote, False for confidence
)

# Properties
vote.voter_id     # Voter ID (bytes)
vote.block_id     # Block ID (bytes)
vote.is_preference # Vote type (bool)

ConsensusEngine

Main consensus engine:

Methods

add_block(block: Block) -> None

Add a block to the consensus engine.

try:
    engine.add_block(block)
except lux.ConsensusError as e:
    print(f"Failed to add block: {e}")

process_vote(vote: Vote) -> None

Process a vote from a validator.

engine.process_vote(vote)

is_accepted(block_id: bytes) -> bool

Check if a block has been accepted.

if engine.is_accepted(block_id):
    print("Block is finalized")

get_preference() -> bytes

Get the currently preferred block ID.

preferred_block = engine.get_preference()

poll(validator_ids: List[bytes]) -> None

Poll a set of validators.

validators = [b'\x01' * 32, b'\x02' * 32, b'\x03' * 32]
engine.poll(validators)

get_stats() -> Stats

Get consensus statistics.

stats = engine.get_stats()
print(f"Blocks: {stats.blocks_accepted} accepted, {stats.blocks_rejected} rejected")
print(f"Polls: {stats.polls_completed} completed")
print(f"Average latency: {stats.average_decision_time_ms:.2f}ms")

Stats

Consensus statistics:

stats.blocks_accepted            # Total blocks accepted (int)
stats.blocks_rejected            # Total blocks rejected (int)
stats.polls_completed            # Total polls completed (int)
stats.votes_processed            # Total votes processed (int)
stats.average_decision_time_ms   # Average decision time in ms (float)

Performance Benchmarks

cd pkg/python
python benchmark_consensus.py

Typical Results (M1 Max):

Block Processing:     ~10,000 blocks/sec
Vote Processing:      ~50,000 votes/sec
Decision Latency:     <1ms average
Memory Usage:         ~100 MB for 10K blocks

Testing

# Run comprehensive tests
cd pkg/python
python -m pytest test_consensus.py -v

# Run with coverage
python -m pytest test_consensus_comprehensive.py --cov=lux_consensus

Error Handling

try:
    engine.add_block(block)
    engine.process_vote(vote)
except lux.ConsensusError as e:
    # Handle consensus-specific errors
    print(f"Consensus error: {e}")
except ValueError as e:
    # Handle validation errors (e.g., wrong ID length)
    print(f"Validation error: {e}")
except MemoryError as e:
    # Handle out-of-memory errors
    print(f"Memory error: {e}")

Examples

Simple Consensus Network

import lux_consensus as lux
import secrets

# Create 5 validator nodes
validators = []
for i in range(5):
    config = lux.ConsensusConfig(
        k=20,
        alpha_preference=3,
        alpha_confidence=3,
        beta=5,
        engine_type=lux.EngineType.DAG
    )
    validators.append(lux.ConsensusEngine(config))

# Propose a block
block = lux.Block(
    block_id=secrets.token_bytes(32),
    parent_id=b'\x00' * 32,
    height=1,
    data=b'Hello, Lux!'
)

# All validators add the block
for validator in validators:
    validator.add_block(block)

# Simulate voting
for i, validator in enumerate(validators):
    vote = lux.Vote(
        voter_id=secrets.token_bytes(32),
        block_id=block.id,
        is_preference=False
    )
    validator.process_vote(vote)

# Check consensus
accepted_count = sum(1 for v in validators if v.is_accepted(block.id))
print(f"{accepted_count}/5 validators accepted the block")

Batch Processing

import lux_consensus as lux
import time

engine = lux.ConsensusEngine(lux.ConsensusConfig())

# Add 1000 blocks
start = time.time()
for i in range(1000):
    block = lux.Block(
        block_id=i.to_bytes(32, 'big'),
        parent_id=(i-1).to_bytes(32, 'big') if i > 0 else b'\x00' * 32,
        height=i
    )
    engine.add_block(block)

elapsed = time.time() - start
stats = engine.get_stats()

print(f"Processed {stats.blocks_accepted} blocks in {elapsed:.2f}s")
print(f"Throughput: {stats.blocks_accepted/elapsed:.0f} blocks/sec")

Advanced Usage

Custom Vote Aggregation

class VoteAggregator:
    def __init__(self, engine):
        self.engine = engine
        self.vote_cache = {}

    def add_vote(self, vote):
        block_id = vote.block_id
        if block_id not in self.vote_cache:
            self.vote_cache[block_id] = []
        self.vote_cache[block_id].append(vote)

        # Process vote
        self.engine.process_vote(vote)

        # Check if consensus reached
        if self.engine.is_accepted(block_id):
            votes = self.vote_cache.pop(block_id, [])
            return True, len(votes)
        return False, len(self.vote_cache.get(block_id, []))

# Usage
aggregator = VoteAggregator(engine)
finalized, vote_count = aggregator.add_vote(vote)
if finalized:
    print(f"Block finalized with {vote_count} votes")

Integration with asyncio

import asyncio
import lux_consensus as lux

async def process_blocks_async(engine, blocks):
    """Process blocks asynchronously"""
    loop = asyncio.get_event_loop()

    for block in blocks:
        # Run in executor to avoid blocking event loop
        await loop.run_in_executor(None, engine.add_block, block)
        await asyncio.sleep(0)  # Yield to event loop

async def main():
    config = lux.ConsensusConfig()
    engine = lux.ConsensusEngine(config)

    blocks = [
        lux.Block(
            block_id=i.to_bytes(32, 'big'),
            parent_id=(i-1).to_bytes(32, 'big') if i > 0 else b'\x00' * 32,
            height=i
        )
        for i in range(100)
    ]

    await process_blocks_async(engine, blocks)
    stats = engine.get_stats()
    print(f"Processed {stats.blocks_accepted} blocks")

asyncio.run(main())

Thread Safety

The Python SDK is thread-safe when using separate engine instances per thread. Shared engine instances require external synchronization:

import threading
import lux_consensus as lux

lock = threading.Lock()
engine = lux.ConsensusEngine(lux.ConsensusConfig())

def worker(block):
    with lock:
        engine.add_block(block)

# Spawn threads
threads = []
for i in range(10):
    block = lux.Block(
        block_id=i.to_bytes(32, 'big'),
        parent_id=b'\x00' * 32,
        height=i
    )
    t = threading.Thread(target=worker, args=(block,))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

Troubleshooting

ImportError: cannot import name 'lux_consensus'

Ensure the C library is installed and the Python package is built:

cd pkg/c && make install
cd ../python && python setup.py build_ext --inplace

ValueError: block_id must be 32 bytes

All IDs must be exactly 32 bytes:

# Correct
block_id = b'\x01' * 32
block_id = bytes.fromhex('01' * 32)
block_id = int(123).to_bytes(32, 'big')

# Wrong
block_id = b'\x01'  # Only 1 byte

MemoryError during batch processing

Reduce max_outstanding_items in config:

config = lux.ConsensusConfig(max_outstanding_items=512)

See Also

  • C SDK - Underlying C library documentation
  • Go SDK - Go implementation with AI consensus
  • Examples - Complete code examples