README

PHP BPE Encoder Decoder for GPT-2/GPT-3

About

GPT-2 and GPT-3 use byte pair encoding to turn text into a series of integers to feed into the model. This is a PHP implementation of OpenAI's original python encoder/decoder which can be found here.

Get Started

Requires PHP 7.4+

Install GPT-3-Encoder via the Composer package manager

composer require rahul900day/gpt-3-encoder

Usage

Encoding a text to tokens

use Rahul900day\Gpt3Encoder\Encoder;

Encoder::encode("Your prompt.");

Decoding to text from tokens

use Rahul900day\Gpt3Encoder\Encoder;

Encoder::decode([8582, 242, 98]);

Credits

• Rahul Dey
• All Contributors

This packages has some codes and test inspiration from node's gpt-3-encoder

License

This package is released under the MIT License.