Facebook makes another machine learning framework open source. Pytext should make it easier to use Natural-Language-Processing in a production environment.
Facebook has released its natural language processing framework Pytext under an open source license. Pytext is based on Pytorch, which also comes from Facebook and is open source. Pytext is used in the social network, for example, for the operation of the portal and the so-called “M recommendations” in Messenger.
Natural language processing deals with the processing of human language. Above all, Facebook’s new framework is designed to help developers use machine learning faster in the development and production environments. According to a blog article, Facebook could bring new machine-learning models from experimental status into productive use within a few days.
NLP development
AI researchers and engineers have a wide and growing range of applications for systems that can understand our words. At Facebook, NLP is used to deliver more relevant content to people, provide powerful accessibility features, flag policy-violating posts, perform translations, and more. The state of the art in conversational AI is progressing rapidly, and PyText is helping us ship these new advancements more quickly, to improve the quality of our products. PyText is now implemented in Portal, our new video calling device, and in our M suggestions feature in Messenger. We are exploring other ways to use PyText in conversational AI. Pytext offers several benefits for NLP development:
- A simplified workflow for faster experimentation.
- Access to a rich set of prebuilt model architectures and utilities for text processing and vocabulary management to facilitate large-scale deployment.
- The ability to harness the PyTorch ecosystem, including prebuilt models and tools created by researchers and engineers in the NLP community.
Pytext is supposed to help with the classification of documents and semantic parsing. It supports so-called multi-task modeling to train several machine-learning models at the same time.
Pytext already performs more than a billion operations a day
Most importantly, the Facebook framework should allow for low-latency work in order to work in real-time, which is one of Facebook’s requirements. In the social network, the framework already performs more than a billion operations a day.
Version 1.0 is now available under the BSD license on GitHub. In addition to the source code, there are already trained models and tutorials to get started.