-
Notifications
You must be signed in to change notification settings - Fork 122
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #760 from NANDAGOPALNG/main
Added a new project called "Text Summarizer"
- Loading branch information
Showing
3 changed files
with
116 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,77 @@ | ||
# Text Summarizer | ||
|
||
A Natural Language Processing (NLP) project that uses Generative AI to summarize long pieces of text into concise, meaningful summaries. | ||
|
||
# Table of Contents | ||
|
||
**Introduction** | ||
|
||
**Features** | ||
|
||
**How it Works** | ||
|
||
**Installation** | ||
|
||
**Usage** | ||
|
||
**Example Use Cases** | ||
|
||
**Contributing** | ||
|
||
**License** | ||
|
||
## Introduction | ||
|
||
The Text Summarizer project uses Gen AI to automatically summarize long pieces of text into shorter, more digestible summaries. This can be useful for a variety of applications, such as: | ||
|
||
Summarizing news articles or blog posts | ||
|
||
Condensing long documents or reports | ||
|
||
Generating abstracts for academic papers | ||
|
||
## Features | ||
|
||
Automatic Summarization: The Text Summarizer uses Gen AI to automatically summarize text without the need for manual intervention. | ||
|
||
Customizable Summary Length: Users can specify the desired length of the summary, from a brief abstract to a longer summary. | ||
|
||
Support for Multiple File Formats: The Text Summarizer can handle text files in various formats, including .txt, .pdf, and .docx. | ||
|
||
## How it Works | ||
|
||
The Text Summarizer uses a combination of natural language processing (NLP) and machine learning algorithms to analyze the input text and generate a summary. The process involves: | ||
|
||
Text Preprocessing: The input text is preprocessed to remove stop words, punctuation, and other irrelevant information. | ||
|
||
Text Analysis: The preprocessed text is then analyzed to identify key phrases, entities, and concepts. | ||
|
||
Summary Generation: The analyzed text is then used to generate a summary, based on the user-specified summary length. | ||
|
||
## Installation | ||
|
||
To install the Text Summarizer, follow these steps: | ||
|
||
Clone the repository: git clone https://github.com/NANDAGOPALNG/ML-Nexus/tree/main/Generative%20Models/Text%20Summarizer | ||
|
||
Install the required dependencies: pip install -r requirements.txt | ||
|
||
Run the Text Summarizer: python text_summarizer.py | ||
|
||
## Usage | ||
|
||
To use the Text Summarizer, simply run the text_summarizer.py script and follow the prompts. You can specify the input text file, summary length, and other options as needed. | ||
|
||
## Example Use Cases | ||
|
||
Summarizing a news article: python text_summarizer.py -i news_article.txt -s 100 | ||
|
||
Summarizing a research paper: python text_summarizer.py -i research_paper.pdf -s 200 | ||
|
||
## Contributing | ||
|
||
Contributions are welcome! If you'd like to contribute to the Text Summarizer project, please fork the repository and submit a pull request. | ||
|
||
## License | ||
|
||
The Text Summarizer project is licensed under the MIT License. See LICENSE for details |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
import torch | ||
import gradio as gr | ||
|
||
# Use a pipeline as a high-level helper | ||
from transformers import pipeline | ||
|
||
text_summary = pipeline("summarization", model="sshleifer/distilbart-cnn-12-6", torch_dtype=torch.bfloat16) | ||
|
||
# model_path = ("../Models/models--sshleifer--distilbart-cnn-12-6/snapshots" | ||
# "/a4f8f3ea906ed274767e9906dbaede7531d660ff") | ||
# text_summary = pipeline("summarization", model=model_path, | ||
# torch_dtype=torch.bfloat16) | ||
|
||
|
||
|
||
# text='''Elon Reeve Musk (/ˈiːlɒn/ EE-lon; born June 28, 1971) is a businessman and investor. | ||
# He is the founder, chairman, CEO, and CTO of SpaceX; angel investor, CEO, product architect, | ||
# and former chairman of Tesla, Inc.; owner, executive chairman, and CTO of X Corp.; | ||
# founder of the Boring Company and xAI; co-founder of Neuralink and OpenAI; and president | ||
# of the Musk Foundation. He is one of the wealthiest people in the world; as of April 2024, | ||
# Forbes estimates his net worth to be $178 billion.[4]''' | ||
# print(text_summary(text)); | ||
|
||
def summary (input): | ||
output = text_summary(input) | ||
return output[0]['summary_text'] | ||
|
||
gr.close_all() | ||
|
||
# demo = gr.Interface(fn=summary, inputs="text",outputs="text") | ||
demo = gr.Interface(fn=summary, | ||
inputs=[gr.Textbox(label="Input text to summarize",lines=6)], | ||
outputs=[gr.Textbox(label="Summarized text",lines=4)], | ||
title="@GenAILearniverse Project 1: Text Summarizer", | ||
description="THIS APPLICATION WILL BE USED TO SUMMARIZE THE TEXT") | ||
demo.launch() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
transformers | ||
torch | ||
gradio |