Skip to content
  • Pepijn de Vos's avatar
    fb7d8cf3
    Replace pickle with msgspec MessagePack serialization (#454) · fb7d8cf3
    Pepijn de Vos authored
    
    
    * Replace pickle with msgspec MessagePack serialization
    
    - Use msgspec for chipdb serialization (faster, typed)
    - Deduplicate grid tiles: grid stores ttyp indices, tiles dict maps to Tile objects
    - Add Device.__getitem__/__setitem__ for grid[row, col] -> Tile access
    - Update all grid iteration patterns across chipdb, gowin_pack, gowin_unpack, etc.
    - Update Makefile, CI workflow, setup.py for .msgpack.gz format
    
    Co-Authored-By: default avatarClaude Opus 4.5 <noreply@anthropic.com>
    
    * Switch chipdb compression from gzip to lzma
    
    LZMA compresses msgpack data ~5x better than gzip (2.8 MB vs 15.8 MB
    total) while decompressing slightly faster. This shrinks the PyPI
    package significantly with no load time penalty.
    
    Co-Authored-By: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    
    * Add cattrs+msgpack fallback for pure Python environments
    
    When msgspec is not available (e.g. PyPy), fall back to cattrs+msgpack
    for deserializing chip databases. Only two custom cattrs hooks needed:
    - Union: cattrs refuses ambiguous unions like Union[int, str]
    - bytearray: msgpack returns bytes, need conversion to bytearray
    
    Also narrow Bel type annotations based on exhaustive data analysis:
    - Bel.flags keys: Dict[Union[int, str], ...] -> Dict[int, ...]
    - Bel.modes keys: Dict[Union[int, str], ...] -> Dict[str, ...]
    
    Co-Authored-By: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    
    * Fix numpy truth-value errors and use [fast] extra in CI
    
    Replace truthiness checks on variables that hold numpy arrays with
    explicit None checks to avoid "ambiguous truth value" ValueError.
    
    Remove the [msgspec] extra (redundant with [fast]) and use [fast]
    for all CI boards except tangnano which tests the [pure] path.
    
    Co-Authored-By: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    
    * Make C extension deps the default, document pure Python alternative
    
    Move msgspec/numpy/crcmod to install_requires so they are installed by
    default. Document `pip install --no-deps apycula msgpack cattrs` in the
    readme for platforms without C extension support.
    
    Simplify CI to match: most boards use default install, tangnano tests
    the pure Python path.
    
    Co-Authored-By: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    
    ---------
    
    Co-authored-by: default avatarClaude Opus 4.5 <noreply@anthropic.com>
    fb7d8cf3
    Replace pickle with msgspec MessagePack serialization (#454)
    Pepijn de Vos authored
    
    
    * Replace pickle with msgspec MessagePack serialization
    
    - Use msgspec for chipdb serialization (faster, typed)
    - Deduplicate grid tiles: grid stores ttyp indices, tiles dict maps to Tile objects
    - Add Device.__getitem__/__setitem__ for grid[row, col] -> Tile access
    - Update all grid iteration patterns across chipdb, gowin_pack, gowin_unpack, etc.
    - Update Makefile, CI workflow, setup.py for .msgpack.gz format
    
    Co-Authored-By: default avatarClaude Opus 4.5 <noreply@anthropic.com>
    
    * Switch chipdb compression from gzip to lzma
    
    LZMA compresses msgpack data ~5x better than gzip (2.8 MB vs 15.8 MB
    total) while decompressing slightly faster. This shrinks the PyPI
    package significantly with no load time penalty.
    
    Co-Authored-By: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    
    * Add cattrs+msgpack fallback for pure Python environments
    
    When msgspec is not available (e.g. PyPy), fall back to cattrs+msgpack
    for deserializing chip databases. Only two custom cattrs hooks needed:
    - Union: cattrs refuses ambiguous unions like Union[int, str]
    - bytearray: msgpack returns bytes, need conversion to bytearray
    
    Also narrow Bel type annotations based on exhaustive data analysis:
    - Bel.flags keys: Dict[Union[int, str], ...] -> Dict[int, ...]
    - Bel.modes keys: Dict[Union[int, str], ...] -> Dict[str, ...]
    
    Co-Authored-By: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    
    * Fix numpy truth-value errors and use [fast] extra in CI
    
    Replace truthiness checks on variables that hold numpy arrays with
    explicit None checks to avoid "ambiguous truth value" ValueError.
    
    Remove the [msgspec] extra (redundant with [fast]) and use [fast]
    for all CI boards except tangnano which tests the [pure] path.
    
    Co-Authored-By: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    
    * Make C extension deps the default, document pure Python alternative
    
    Move msgspec/numpy/crcmod to install_requires so they are installed by
    default. Document `pip install --no-deps apycula msgpack cattrs` in the
    readme for platforms without C extension support.
    
    Simplify CI to match: most boards use default install, tangnano tests
    the pure Python path.
    
    Co-Authored-By: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    
    ---------
    
    Co-authored-by: default avatarClaude Opus 4.5 <noreply@anthropic.com>
Loading