Pipfeed's Extract API Developer Documentation

Extract data from online news & articles. Get full metadata with content, images, authors, summary, category, keywords, topics, and more.

Welcome to the developer documentation for Pipfeed's Extract API. This API provides capabilities for extracting detailed information from news articles, including the title, author, content, images, and more.

Developer Portal: https://app.pipfeed.com/

The most advanced article extraction API with AI/ML summary, category prediction, all images, blog logo, authors, keywords, tags, and more.

Avoid months of development time building your custom URL extractor

We have spent months fine-tuning each small part of our article extract API so you don't have to. One API can help you achieve various use cases. These are some of the use-cases our customers Pipfeed's API help enable:

  • Add "Reader Mode" to your apps.

  • Send "summarized" content to your readers.

  • Use parsed data in training AI models.

  • Allow readers to download the content and read it offline.

API Documentation

Endpoint

  • URL: https://api.magicapi.com/api/v1/pipfeed/{plan_name}/extract

  • Method: POST

Headers:

  • x-magicapi-key: Your unique API key. Note: Do not share this key publicly.

  • Content-Type: Must be set to application/json.

Authentication

To use the Pipfeed's Extract API, you must include your API key in the request header.

Header Name: x-magicapi-key Value: your-api-key

You can get an API by signing up at https://app.pipfeed.com. If you just want to test then sign up for the "FREE(no card)" plan. This plan allows you to send 100 API requests per month at no charge or credit card.

Request Payload

  • url (required): The URL of the article you wish to extract data from.

{
  "url": "https://example.com/news-article"
}

Response Data

The API will return the following data:

  • url: The original URL of the article.

  • title: The title of the article.

  • author: The author's name.

  • html: The article's HTML content.

  • text: The plaintext version of the article.

  • length: The number of characters in the text content.

  • description: A brief description or sub-title of the article.

  • siteName: The website or source name.

  • topImage: The URL of the main image of the article.

  • date: The published date of the article.

  • keywords: Keywords related to the article.

  • summary: An array of summarized points or sentences from the article.

  • sentiment: A sentiment analysis of the article's content.

Sentiment Analysis

The sentiment analysis object contains:

  • score: A numerical score representing the overall sentiment. Positive numbers indicate positive sentiment, and negative numbers indicate negative sentiment.

  • comparative: A normalized score by length.

  • calculation: An array of words and their corresponding sentiment scores.

  • positive: An array of positive words found in the article.

  • negative: An array of negative words found in the article.

Example cURL Request

curl --location 'https://api.magicapi.com/api/v1/pipfeed/free_no_card/extract' \
--header 'x-magicapi-key: your-api-key' \
--header 'Content-Type: application/json' \
--data '{
    "url": "https://www.nytimes.com/2023/07/05/world/europe/russia-ukraine-prisoner-interview.html"
}'

Note: Replace x-magicapi-key with your actual MagicAPI key.

Example JavaScript Fetch Request


let parseUrl = "https://www.nytimes.com/2023/07/05/world/europe/russia-ukraine-prisoner-interview.html";
var raw = JSON.stringify({
  "url": parseUrl
});

const url = 'https://api.magicapi.dev/api/v1/pipfeed/pro/extract';
const options = {
	method: 'POST',
	headers: {
		'content-type': 'application/json',
	        'x-magicapi-key': 'your-api-key',
	},
	body: raw
};

try {
	const response = await fetch(url, options);
	const result = await response.text();
	console.log(result);
} catch (error) {
	console.error(error);
}

Considerations

  • Always keep your MagicAPI key confidential.

  • Respect the terms of use for the websites you are extracting information from.

  • Ensure you handle the data you obtain responsibly and ethically.

By utilizing this API, you can quickly gather rich data from articles and further analyze or display the extracted information as per your application's needs.

NextFAQs

Last updated