/

sentencepiece

sentencepiece

Owned by Niyas Sait (Deactivated)

Last updated: May 23, 2022

1 min read

GitHub - google/sentencepiece: Unsupervised text tokenizer for Neural Network-based text generation.

Steps to build sentencepiece library and python package is given below.

git clone https://github.com/google/sentencepiece.git

cd sentencepiece

mkdir build

cd build

set CURRENT_PATH=%~dp0

set LIBRARY_PATH=%CURRENT_PATH%build\root

cmake .. -A ARM64 -DSPM_BUILD_TEST=ON -DSPM_ENABLE_SHARED=OFF -DCMAKE_INSTALL_PREFIX=%LIBRARY_PATH%

cmake --build . --config Release --target install

cd ../python

python setup.py bdist_wheel

Related content

Pre-release Python binary wheels and dependencies

Pre-release Python binary wheels and dependencies

Windows On Arm (WOA)

More like this

Shapely and PyGEOS

Shapely and PyGEOS

Windows On Arm (WOA)

More like this

Pillow

Windows On Arm (WOA)

More like this

CMake

Windows On Arm (WOA)

More like this

PyWin32

Windows On Arm (WOA)

More like this

Pyproj

Windows On Arm (WOA)

More like this