{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/6"}],"enrichment":{"capability":"Modifies Hugging Face Transformers model.generate() to return a token-by-token generator instead of waiting for full inference completion, enabling real-time streaming output during text generation.","skillfed_tags":["streaming-inference","monkey-patching"],"use_cases":["Build a chatbot or conversational interface that displays model responses token-by-token as they are generated.","Stream long-form text generation (summaries, articles) to a web client in real-time without waiting for completion.","Implement live transcription or translation where partial results appear as soon as each token is ready.","Create interactive demos where users see model output appearing incrementally, improving perceived responsiveness."],"what_it_does":"This package wraps Hugging Face Transformers to enable token-by-token streaming during text generation. Instead of waiting for model.generate() to complete and return all tokens at once, you call init_stream_support() once, then pass do_stream=True to generate(), and receive a generator that yields each token as it is produced. This is useful for real-time applications like chatbots or live transcription where you want to display output incrementally rather than after the full inference completes.\n\nThe package works by modifying Transformers internals through monkey-patching. It has no explicit runtime dependencies and supports Python 3.5 and later. However, it is dormant (last updated 2024-03-11) and may not work with newer Transformers versions that have changed their generation API or internal structure.","worth_installing":"Yes, with conditions. Install if you need token-by-token streaming for a Transformers model and your Transformers version is close to 2024-03-11 or earlier. Do not install if you are using a recent Transformers version, as the dormant status and monkey-patching approach mean compatibility is not guaranteed. Test thoroughly in your environment before production use."},"id":"transformers-stream-generator","links":{"html":"https://skillfed.io/packages/transformers-stream-generator","md":"https://skillfed.io/packages/transformers-stream-generator.md","pypi":"https://pypi.org/project/transformers-stream-generator/"},"maintenance":{"status":"dormant"},"meta":{"latest_release":"2024-03-11","license_spdx":null,"license_treatment":"permissive","name":"transformers-stream-generator","python_support":"supports_current","summary":"This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/Transformers."},"popularity":{"monthly_downloads":345880,"position":7360,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.0.5"}