Repository: Benjamin-Loison/YouTube-operational-API
Branch: main
Commit: 0d2768a5fcf5
Files: 34
Total size: 172.1 KB
Directory structure:
gitextract_ugyzn8kf/
├── .gitignore
├── .htaccess
├── CITATION.cff
├── CONTRIBUTING.md
├── Dockerfile
├── README.md
├── addKey.php
├── channels.php
├── commentThreads.php
├── common.php
├── community.php
├── configuration.php
├── constants.php
├── docker-compose.yml
├── index.php
├── keys.php
├── liveChats.php
├── lives.php
├── noKey/
│ ├── .htaccess
│ └── index.php
├── playlistItems.php
├── playlists.php
├── proto/
│ ├── php/
│ │ └── .gitignore
│ └── prototypes/
│ ├── browse.proto
│ └── browse_shorts.proto
├── search.php
├── tools/
│ ├── checkOperationnalAPI.py
│ ├── getJSONPathFromKey.py
│ ├── minimizeCURL.py
│ └── simplifyCURL.py
├── videos.php
└── ytPrivate/
├── .htaccess
├── test.php
└── tests.php
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
.env
ytPrivate/keys.txt
================================================
FILE: .htaccess
================================================
Options +FollowSymLinks
RewriteEngine on
RewriteRule ^search$ search.php
RewriteRule ^videos$ videos.php
RewriteRule ^playlists$ playlists.php
RewriteRule ^playlistItems$ playlistItems.php
RewriteRule ^channels$ channels.php
RewriteRule ^community$ community.php
RewriteRule ^webhooks$ webhooks.php
RewriteRule ^commentThreads$ commentThreads.php
RewriteRule ^lives$ lives.php
RewriteRule ^liveChats$ liveChats.php
<IfModule mod_headers.c>
Header set Access-Control-Allow-Origin "*"
</IfModule>
# For official instances:
Redirect /matrix https://matrix.to/#/#youtube-operational-api:matrix.org
Redirect /discord https://discord.gg/pDzafhGWzf
Redirect /code https://github.com/Benjamin-Loison/YouTube-operational-API
Redirect /host-your-own-instance https://github.com/Benjamin-Loison/YouTube-operational-API/blob/main/README.md#install-your-own-instance-of-the-api
Redirect /issues https://github.com/Benjamin-Loison/YouTube-operational-API/issues
================================================
FILE: CITATION.cff
================================================
cff-version: 1.2.0
title: YouTube operational API
message: >-
If you use this software, please cite it using the
metadata from this file. Please let me know if you cite
this software.
type: software
authors:
- given-names: Benjamin
family-names: Loison
repository-code: 'https://github.com/Benjamin-Loison/YouTube-operational-API'
abstract: >-
YouTube operational API works when YouTube Data API v3
fails.
keywords:
- YouTube
- YouTube Data API v3
- YouTube operational API
================================================
FILE: CONTRIBUTING.md
================================================
# Welcome to the YouTube operational API contributing guide
Thank you for investing your time in contributing to YouTube operational API!
**Table of contents:**
- [Contributing as a user](#contributing-as-a-user)
- [Contributing as a developer](#contributing-as-a-developer)
## Contributing as a user
If you encounter any problem or have any suggestion, don't hesitate to [open an issue](https://github.com/Benjamin-Loison/YouTube-operational-API/issues) if your problem isn't already listed. Don't hesitate to [reach out the YouTube operational API community](https://github.com/Benjamin-Loison/YouTube-operational-API/#contact).
## Contributing as a developer
In addition to [possibilities to contribute as a user](#contributing-as-a-user), if you could try to address [already listed issues](https://github.com/Benjamin-Loison/YouTube-operational-API/issues) through [pull requests](https://github.com/Benjamin-Loison/YouTube-operational-API/pulls), it would be greatly appreciated!
================================================
FILE: Dockerfile
================================================
FROM php:apache
RUN a2enmod rewrite
# Copy application files into the container
COPY . /var/www/html/
# Replace `AllowOverride None` with `AllowOverride All` in `<Directory /var/www/>` in `/etc/apache2/apache2.conf`.
RUN sed -ri -e 'N;N;N;s/(<Directory \/var\/www\/>\n)(.*\n)(.*)AllowOverride None/\1\2\3AllowOverride All/;p;d;' /etc/apache2/apache2.conf
COPY --from=composer/composer:latest-bin /composer /usr/bin/composer
RUN apt update
RUN apt install -y git protobuf-compiler
RUN composer require google/protobuf
RUN protoc --php_out=proto/php/ --proto_path=proto/prototypes/ $(find proto/prototypes/ -type f)
CMD apachectl -D FOREGROUND
================================================
FILE: README.md
================================================
# YouTube operational API
YouTube operational API works when [YouTube Data API v3](https://developers.google.com/youtube/v3) fails.
## Install your own instance of the API:
1. If you do not own a server, then I recommend Oracle always free VPS (see https://docs.oracle.com/en-us/iaas/Content/FreeTier/freetier_topic-Always_Free_Resources.htm). Note that at account creation only some *home regions*, including *Marseille* (France), propose more powerful *Ampere A1* shapes.
2. If not already hosting a website ([click here](https://github.com/Benjamin-Loison/YouTube-operational-API/wiki/Home/1c7139f68af217d41d0a201a97eaecf87c139a8b#install-your-own-instance-of-the-api-on-a-nginx-web-server) if you prefer nginx), run in a terminal:
### On Linux (Debian, Mint and Ubuntu):
```
sudo apt install apache2 php git
sudo a2enmod rewrite headers
```
Replace `AllowOverride None` with `AllowOverride All` in `<Directory /var/www/>` in `/etc/apache2/apache2.conf`.
Then run:
```
sudo service apache2 restart
```
### On Windows:
Download and run [WampServer 3](https://sourceforge.net/projects/wampserver/files/latest/download).
### On MacOS:
Install `brew` by following https://brew.sh#install.
On MacOS (Intel) use `/usr/local/` instead of `/opt/homebrew/`.
```zsh
brew install apache2 php
echo 'LoadModule php_module /opt/homebrew/opt/php/lib/httpd/modules/libphp.so
Include /opt/homebrew/etc/httpd/extra/httpd-php.conf' >> /opt/homebrew/etc/httpd/httpd.conf
echo '<IfModule php_module>
<FilesMatch \.php$>
SetHandler application/x-httpd-php
</FilesMatch>
<IfModule dir_module>
DirectoryIndex index.php
</IfModule>
</IfModule>' >> /opt/homebrew/etc/httpd/extra/httpd-php.conf
sed -i '' 's/#LoadModule rewrite_module/LoadModule rewrite_module/' /opt/homebrew/etc/httpd/httpd.conf
```
Replace `AllowOverride None` with `AllowOverride All` in `<Directory "/opt/homebrew/var/www">` in `/opt/homebrew/etc/httpd/httpd.conf`.
Then run:
```
brew services start httpd
```
3. Now that you are hosting a website, get the current working directory of your terminal into the folder that is online.
- On Linux, use `cd /var/www/html/`
- On Windows, use `cd C:\wamp64\www\`
- On MacOS, use `cd /opt/homebrew/var/www/`
4. Clone this repository by using:
```sh
git clone https://github.com/Benjamin-Loison/YouTube-operational-API
```
5. Install Protobuf dependency:
### On Linux (Ubuntu, Debian and Mint):
```sh
sudo apt install composer protobuf-compiler
```
### On Windows:
Download [composer](https://github.com/composer/windows-setup/releases/latest).
Download [protoc](https://github.com/protocolbuffers/protobuf/releases/latest).
### On MacOS:
```sh
brew install composer protobuf
```
In `YouTube-operational-API/` clone folder:
```sh
composer require google/protobuf
```
Generate code of PHP objects from `.proto` prototypes:
### On Linux and MacOS:
```sh
protoc --php_out=proto/php/ --proto_path=proto/prototypes/ $(find proto/prototypes/ -type f)
```
### On Windows:
```batch
for /f "usebackq tokens=*" %a in (`dir /S /B "proto/prototypes"`) do protoc --php_out=proto/php/ --proto_path=proto/prototypes/ %a
```
6. Verify that your API instance is reachable by trying to access:
- On Linux and Windows: http://localhost/YouTube-operational-API/
- On MacOS: http://localhost:8080/YouTube-operational-API/
If you want me to advertise your instance (if you have opened your port, and have a fixed IP address or a domain name), please use below contacts.
## Run the API with Docker
1. Install [Docker](https://www.docker.com) and make sure that its daemon is running.
2. Create a `.env` file and update it with your preferred port:
```sh
cp .env.sample .env
```
3. Start the container with `docker-compose`:
```sh
# start in the foreground
docker-compose up
# start in the background
docker-compose up -d
```
4. Verify that your API instance is reachable by trying to access:
- http://localhost:8080 (update preferred port if not 8080)
## Contact:
- [Matrix](https://yt.lemnoslife.com/matrix)
- [Discord](https://yt.lemnoslife.com/discord)
## Contributing:
See [`CONTRIBUTING.md`](https://github.com/Benjamin-Loison/YouTube-operational-API/blob/main/CONTRIBUTING.md).
================================================
FILE: addKey.php
================================================
<?php
include_once 'common.php';
if (isset($_GET['key'])) {
$key = $_GET['key'];
// Regex-based filter.
if (isYouTubeDataAPIV3Key($key)) {
$keysContent = file_get_contents(KEYS_FILE);
$keys = explode("\n", $keysContent);
// Verify that the YouTube Data API v3 key isn't already stored by the instance.
if (!in_array($key, $keys)) {
$httpOptions = [
'http' => [
'ignore_errors' => true,
]
];
$content = getJSON("https://www.googleapis.com/youtube/v3/videos?part=snippet&id=mWdFMNQBcjs&key=$key", $httpOptions, false);
// The force secret is used to store the YouTube Data API v3 even if it's not having quota, as we assume that the trusted instance that send it to this one has checked that it has quota.
if ($content['items'][0]['snippet']['title'] === 'A public video' || (isset($_GET['forceSecret']) && $_GET['forceSecret'] === ADD_KEY_FORCE_SECRET)) {
file_put_contents(KEYS_FILE, ($keysContent === '' || $keysContent === false ? '' : "\n") . $key, FILE_APPEND);
// Avoid sending another time the given key to all instances.
if (!isset($_GET['forceSecret'])) {
foreach (ADD_KEY_TO_INSTANCES as $addKeyToInstance) {
getRemote($addKeyToInstance . "addKey.php?key=$key&forceSecret=" . ADD_KEY_FORCE_SECRET);
}
}
echo 'YouTube Data API v3 key added.';
} elseif ($content['error']['errors'][0]['reason'] === 'quotaExceeded') {
// As users can set `Queries per minute` quota to 0, we avoid denial-of-service by not considering them.
echo 'Not adding YouTube Data API v3 key having quota exceeded.';
} else {
// This YouTube Data API API v3 keys isn't assigned or there is another error.
echo 'Incorrect YouTube Data API v3 key.';
}
} else {
echo 'This YouTube Data API v3 key is already in the list.';
}
} else {
echo "The key provided isn't a YouTube Data API v3 key.";
}
}
================================================
FILE: channels.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
include_once 'common.php';
$channelsTests = [
['cId=FolkartTr', 'items/0/id', 'UCnS--2e1yzQCm5r4ClrMJBg'],
['handle=@Test-kq9ig', 'items/0/id', 'UCv_LqFI-0vMVYgNR3TeB3zQ'],
['forUsername=DonDiablo', 'items/0/id', 'UC8y7Xa0E1Lo6PnVsu2KJbOA'],
['part=status&id=UC7LoiySz7-FcGgZCKBq_2vQ', 'items/0/status', 'This channel is not available.'],
// How to precise viewCount can be any integer greater than those we have? Same concerning relative date.
// Do not forget to format JSON
['part=shorts&id=UCv_LqFI-0vMVYgNR3TeB3zQ', 'items/0/shorts', json_decode(file_get_contents('tests/part=shorts&id=UCv_LqFI-0vMVYgNR3TeB3zQ.json'), true)],
['part=community&id=UCv_LqFI-0vMVYgNR3TeB3zQ', 'items/0/community', json_decode(file_get_contents('tests/part=community&id=UCv_LqFI-0vMVYgNR3TeB3zQ.json'), true)],
['part=about&id=UCv_LqFI-0vMVYgNR3TeB3zQ', 'items/0', json_decode(file_get_contents('tests/part=about&id=UCv_LqFI-0vMVYgNR3TeB3zQ.json'), true)],
['part=approval&id=UC0aMaqIs997ggjDs_Q9UYiw', 'items/0/approval', 'Official Artist Channel'],
['part=snippet&id=UCv_LqFI-0vMVYgNR3TeB3zQ', 'items/0/snippet', json_decode(file_get_contents('tests/part=snippet&id=UCv_LqFI-0vMVYgNR3TeB3zQ.json'), true)],
['part=membership&id=UCX6OQ3DkcsbYNE6H8uQQuVA', 'items/0/isMembershipEnabled', true],
['part=popular&id=UCyvTYozFRVuM_mKKyT6K50g', 'items/0', []],
['part=recent&id=UCyvTYozFRVuM_mKKyT6K50g', 'items/0', []],
['part=letsPlay&id=UCyvTYozFRVuM_mKKyT6K50g', 'items/0', []],
];
$realOptions = [
'status',
'upcomingEvents',
'shorts',
'community',
'channels',
'about',
'approval',
'playlists',
'snippet',
'membership',
'popular',
'recent',
'letsPlay',
];
// really necessary ?
foreach ($realOptions as $realOption) {
$options[$realOption] = false;
}
// Forbidding URL with no `part` and using `id` filter is debatable.
if (isset($_GET['cId']) || isset($_GET['id']) || isset($_GET['handle']) || isset($_GET['forUsername']) || isset($_GET['raw'])) {
if(isset($_GET['part'])) {
$part = $_GET['part'];
$parts = explode(',', $part, count($realOptions));
foreach ($parts as $part) {
if (!in_array($part, $realOptions)) {
dieWithJsonMessage("Invalid part $part");
} else {
$options[$part] = true;
}
}
}
$ids = [];
if (isset($_GET['cId'])) {
$realCIds = getMultipleIds('cId');
foreach ($realCIds as $realCId)
{
if (!isCId($realCId)) {
dieWithJsonMessage('Invalid cId');
}
$result = getJSONFromHTML("https://www.youtube.com/c/$realCId/about");
$id = $result['metadata']['channelMetadataRenderer']['externalId'];
array_push($ids, $id);
}
} else if (isset($_GET['id'])) {
$realIds = getMultipleIds('id');
foreach ($realIds as $realId)
{
if (!isChannelId($realId)) {
dieWithJsonMessage('Invalid id');
}
}
$ids = $realIds;
} else if (isset($_GET['handle'])) {
$realHandles = getMultipleIds('handle');
foreach ($realHandles as $realHandle)
{
if (!isHandle($realHandle)) {
dieWithJsonMessage('Invalid handle');
}
$result = getJSONFromHTML("https://www.youtube.com/$realHandle");
$params = $result['responseContext']['serviceTrackingParams'][0]['params'];
foreach($params as $param)
{
if($param['key'] === 'browse_id')
{
$id = $param['value'];
break;
}
}
array_push($ids, $id);
}
}
else if (isset($_GET['forUsername'])) {
$realUsernames = getMultipleIds('forUsername');
foreach ($realUsernames as $realUsername)
{
if (!isUsername($realUsername)) {
dieWithJsonMessage('Invalid forUsername');
}
$result = getJSONFromHTML("https://www.youtube.com/user/$realUsername");
$id = $result['header']['c4TabbedHeaderRenderer']['channelId'];
array_push($ids, $id);
}
}
else /*if (isset($_GET['raw']))*/ {
$realRaws = getMultipleIds('raw');
foreach ($realRaws as $realRaw)
{
// Adding filter would be nice.
$result = getJSONFromHTML("https://www.youtube.com/$realRaw");
$id = $result['header']['c4TabbedHeaderRenderer']['channelId'];
array_push($ids, $id);
}
}
$order = 'time';
if (isset($_GET['order'])) {
$order = $_GET['order'];
if (!in_array($order, ['time', 'viewCount'])) {
dieWithJsonMessage('Invalid order');
}
}
$continuationToken = '';
if (isset($_GET['pageToken'])) {
$continuationToken = $_GET['pageToken'];
$hasVisitorData = $options['shorts'] || $options['popular'] || $options['recent'];
if (($hasVisitorData && !isContinuationTokenAndVisitorData($continuationToken)) || (!$hasVisitorData && !isContinuationToken($continuationToken))) {
dieWithJsonMessage('Invalid pageToken');
}
}
echo getAPI($ids, $order, $continuationToken);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function getItem($id, $order, $continuationToken)
{
global $options;
$item = [
'kind' => 'youtube#channel',
'etag' => 'NotImplemented',
'id' => $id
];
$continuationTokenProvided = $continuationToken != '';
if ($options['status']) {
$result = getJSONFromHTML("https://www.youtube.com/channel/$id", forceLanguage: true, verifiesChannelRedirection: true);
$status = $result['alerts'][0]['alertRenderer']['text']['simpleText'];
$item['status'] = $status;
}
if ($options['upcomingEvents']) {
$upcomingEvents = [];
$result = getJSONFromHTML("https://www.youtube.com/channel/$id", forceLanguage: true, verifiesChannelRedirection: true);
$subItems = getTabs($result)[0]['tabRenderer']['content']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'][0]['shelfRenderer']['content']['horizontalListRenderer']['items'];
foreach ($subItems as $subItem) {
$path = 'gridVideoRenderer/upcomingEventData';
if (doesPathExist($subItem, $path)) {
$subItem = $subItem['gridVideoRenderer'];
foreach (['navigationEndpoint', 'menu', 'trackingParams', 'thumbnailOverlays'] as $toRemove) {
unset($subItem[$toRemove]);
}
array_push($upcomingEvents, $subItem);
}
}
$item['upcomingEvents'] = $upcomingEvents;
}
if ($options['shorts']) {
if (!$continuationTokenProvided) {
$result = getJSONFromHTML("https://www.youtube.com/channel/$id/shorts", forceLanguage: true, verifiesChannelRedirection: true);
$visitorData = getVisitorData($result);
$tab = getTabByName($result, 'Shorts');
$tabRenderer = $tab['tabRenderer'];
$richGridRenderer = $tabRenderer['content']['richGridRenderer'];
if ($order === 'viewCount') {
$nextPageToken = $richGridRenderer['header']['feedFilterChipBarRenderer']['contents'][1]['chipCloudChipRenderer']['navigationEndpoint']['continuationCommand']['token'];
if($nextPageToken !== null) {
$continuationToken = urldecode("$nextPageToken,$visitorData");
return getItem($id, $order, $continuationToken);
}
}
} else {
$result = getContinuationJson($continuationToken);
}
$shorts = [];
if (!$continuationTokenProvided) {
$reelShelfRendererItems = $richGridRenderer['contents'];
}
else {
$onResponseReceivedActions = $result['onResponseReceivedActions'];
$onResponseReceivedAction = $onResponseReceivedActions[count($onResponseReceivedActions) - 1];
$continuationItems = getValue($onResponseReceivedAction, 'appendContinuationItemsAction', 'reloadContinuationItemsCommand');
$reelShelfRendererItems = $continuationItems['continuationItems'];
}
foreach($reelShelfRendererItems as $reelShelfRendererItem) {
if(!array_key_exists('richItemRenderer', $reelShelfRendererItem))
continue;
$shortsLockupViewModel = $reelShelfRendererItem['richItemRenderer']['content']['shortsLockupViewModel'];
$overlayMetadata = $shortsLockupViewModel['overlayMetadata'];
$reelWatchEndpoint = $shortsLockupViewModel['onTap']['innertubeCommand']['reelWatchEndpoint'];
$short = [
'videoId' => $reelWatchEndpoint['videoId'],
'viewCount' => getIntValue($overlayMetadata['secondaryText']['content'], 'view'),
'title' => $overlayMetadata['primaryText']['content'],
// Both `sqp` and `rs` parameters are required to crop correctly the thumbnail.
'thumbnail' => $shortsLockupViewModel['thumbnail']['sources'][0],
'frame0Thumbnail' => $reelWatchEndpoint['thumbnail']['thumbnails'],
];
if (!$continuationTokenProvided) {
$browseEndpoint = $tabRenderer['endpoint']['browseEndpoint'];
$short['channelHandle'] = substr($browseEndpoint['canonicalBaseUrl'], 1);
$short['channelId'] = $browseEndpoint['browseId'];
}
array_push($shorts, $short);
}
$item['shorts'] = $shorts;
if($reelShelfRendererItems != null && count($reelShelfRendererItems) > 48) {
$nextPageToken = $reelShelfRendererItems[48]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'];
$item['nextPageToken'] = urldecode("$nextPageToken,$visitorData");
}
}
if ($options['community']) {
if (!$continuationTokenProvided) {
define('COMMUNITY_TAB_NAME', 'Posts');
$result = getJSONFromHTML("https://www.youtube.com/channel/$id/" . strtolower(COMMUNITY_TAB_NAME), forceLanguage: true, verifiesChannelRedirection: true);
} else {
$result = getContinuationJson($continuationToken);
}
$community = [];
$contents = null;
if (!$continuationTokenProvided) {
$tab = getTabByName($result, COMMUNITY_TAB_NAME);
$contents = $tab['tabRenderer']['content']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'];
} else {
$contents = $result['onResponseReceivedEndpoints'][0]['appendContinuationItemsAction']['continuationItems'];
}
foreach ($contents as $content) {
// What is the purpose of this condition?
if (!array_key_exists('backstagePostThreadRenderer', $content)) {
continue;
}
$post = getCommunityPostFromContent($content);
array_push($community, $post);
}
$item['community'] = $community;
if ($contents !== null && array_key_exists('continuationItemRenderer', end($contents))) {
$item['nextPageToken'] = urldecode(end($contents)['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token']);
}
}
if ($options['channels']) {
if (!$continuationTokenProvided) {
$result = getJSONFromHTML("https://www.youtube.com/channel/$id/channels", forceLanguage: true, verifiesChannelRedirection: true);
$tab = getTabByName($result, 'Channels');
$sectionListRenderer = $tab['tabRenderer']['content']['sectionListRenderer'];
$contents = array_map(fn($content) => $content['itemSectionRenderer']['contents'][0], $sectionListRenderer['contents']);
$itemsArray = [];
foreach($contents as $content)
{
if (array_key_exists('shelfRenderer', $content)) {
$sectionTitle = $content['shelfRenderer']['title']['runs'][0]['text'];
$content = $content['shelfRenderer']['content'];
$content = getValue($content, 'horizontalListRenderer', 'expandedShelfContentsRenderer');
} else {
$sectionTitle = $sectionListRenderer['subMenu']['channelSubMenuRenderer']['contentTypeSubMenuItems'][0]['title'];
$content = $content['gridRenderer'];
}
array_push($itemsArray, [$sectionTitle, $content['items']]);
}
} else {
$result = getContinuationJson($continuationToken);
$itemsArray = [[null, getContinuationItems($result)]];
}
$channelSections = [];
foreach($itemsArray as [$sectionTitle, $items]) {
$sectionChannels = [];
$nextPageToken = null;
$lastChannelItem = !empty($items) ? end($items) : [];
$path = 'continuationItemRenderer/continuationEndpoint/continuationCommand/token';
if (doesPathExist($lastChannelItem, $path)) {
$nextPageToken = urldecode(getValue($lastChannelItem, $path));
$items = array_slice($items, 0, count($items) - 1);
}
foreach($items as $sectionChannelItem) {
$gridChannelRenderer = $sectionChannelItem[getValue($sectionChannelItem, 'gridChannelRenderer', 'channelRenderer')];
// Condition required for channel `UC-1BnotsIsigEK4zLw20IDQ` which doesn't have a `CHANNELS` tab and using the `channel/CHANNEL_ID/channels` URL shows the `HOME` channel tab content.
if($gridChannelRenderer === null) {
goto breakChannelSectionsTreatment;
}
$thumbnails = [];
foreach($gridChannelRenderer['thumbnail']['thumbnails'] as $thumbnail) {
$thumbnail['url'] = 'https://' . substr($thumbnail['url'], 2);
array_push($thumbnails, $thumbnail);
}
$subscriberCount = getIntValue($gridChannelRenderer['subscriberCountText']['simpleText'], 'subscriber');
// Have observed the singular case for the channel: https://www.youtube.com/channel/UCbOoDorgVGd-4vZdIrU4C1A
$channel = [
'channelId' => $gridChannelRenderer['channelId'],
'title' => $gridChannelRenderer['title']['simpleText'],
'thumbnails' => $thumbnails,
'videoCount' => intval(str_replace(',', '', $gridChannelRenderer['videoCountText']['runs'][0]['text'])),
'subscriberCount' => $subscriberCount
];
array_push($sectionChannels, $channel);
}
array_push($channelSections, [
'title' => $sectionTitle,
'sectionChannels' => $sectionChannels,
'nextPageToken' => $nextPageToken
]);
}
breakChannelSectionsTreatment:
$item['channelSections'] = $channelSections;
}
if ($options['about']) {
$result = getJSONFromHTML("https://www.youtube.com/channel/$id/about", forceLanguage: true, verifiesChannelRedirection: true);
$c4TabbedHeaderRenderer = $result['header']['c4TabbedHeaderRenderer'];
$item['countryChannelId'] = $c4TabbedHeaderRenderer['channelId'];
$tab = getTabByName($result, 'About');
$resultCommon = $result['onResponseReceivedEndpoints'][0]['showEngagementPanelEndpoint']['engagementPanel']['engagementPanelSectionListRenderer']['content']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'][0]['aboutChannelRenderer']['metadata']['aboutChannelViewModel'];
$about['stats'] = [
'joinedDate' => strtotime(str_replace('Joined ', '', $resultCommon['joinedDateText']['content'])),
// Could try to find a YouTube channel with a single view to make sure it displays "view" and not "views".
'viewCount' => getIntValue($resultCommon['viewCountText'], 'view'),
'subscriberCount' => getIntValue($c4TabbedHeaderRenderer['subscriberCountText']['simpleText'], 'subscriber'),
'videoCount' => getIntValue($resultCommon['videoCountText'], 'video')
];
$about['description'] = $resultCommon['description'];
$about['details'] = [
'location' => $resultCommon['country']
];
$linksObjects = $resultCommon['links'];
$links = [];
foreach ($linksObjects as $linkObject) {
$linkObject = $linkObject['channelExternalLinkViewModel'];
$url = $linkObject['link']['commandRuns'][0]['onTap']['innertubeCommand']['urlEndpoint']['url'];
$urlComponents = parse_url($url);
parse_str($urlComponents['query'], $params);
$link = [
'url' => getValue($params, 'q', defaultValue: $url),
'title' => $linkObject['title']['content'],
'favicon' => $linkObject['favicon']['sources']
];
array_push($links, $link);
}
$about['links'] = $links;
$about['handle'] = substr($result['contents']['twoColumnBrowseResultsRenderer']['tabs'][0]['tabRenderer']['endpoint']['browseEndpoint']['canonicalBaseUrl'], 1);
$item['about'] = $about;
}
if ($options['approval']) {
$result = getJSONFromHTML("https://www.youtube.com/channel/$id", forceLanguage: true, verifiesChannelRedirection: true);
$item['approval'] = end(explode(', ', $result['header']['pageHeaderRenderer']['content']['pageHeaderViewModel']['title']['dynamicTextViewModel']['rendererContext']['accessibilityContext']['label']));
}
if ($options['snippet']) {
$result = getJSONFromHTML("https://www.youtube.com/channel/$id", verifiesChannelRedirection: true);
$c4TabbedHeaderRenderer = $result['header']['c4TabbedHeaderRenderer'];
$c4TabbedHeaderRendererKeys = ['avatar', 'banner', 'tvBanner', 'mobileBanner'];
$snippet = array_combine($c4TabbedHeaderRendererKeys, array_map(fn($c4TabbedHeaderRendererKey) => $c4TabbedHeaderRenderer[$c4TabbedHeaderRendererKey]['thumbnails'], $c4TabbedHeaderRendererKeys));
$item['snippet'] = $snippet;
}
if ($options['membership']) {
$result = getJSONFromHTML("https://www.youtube.com/channel/$id");
$item['isMembershipEnabled'] = doesPathExist($result, 'header/c4TabbedHeaderRenderer/sponsorButton') || getValue($result, 'header/pageHeaderRenderer/content/pageHeaderViewModel/actions/flexibleActionsViewModel/actionsRows/0/actions/1/buttonViewModel/targetId', $defaultValue = null) === 'sponsorships-button';
}
if ($options['playlists']) {
if (!$continuationTokenProvided) {
$result = getJSONFromHTML("https://www.youtube.com/channel/$id/playlists", forceLanguage: true, verifiesChannelRedirection: true);
$tab = getTabByName($result, 'Playlists');
if ($tab === null) {
die(returnItems([]));
}
$sectionListRenderer = $tab['tabRenderer']['content']['sectionListRenderer'];
$contents = array_map(fn($content) => $content['itemSectionRenderer']['contents'][0], $sectionListRenderer['contents']);
$itemsArray = [];
foreach($contents as $content)
{
if (array_key_exists('shelfRenderer', $content)) {
$sectionTitle = $content['shelfRenderer']['title']['runs'][0]['text'];
$content = $content['shelfRenderer']['content'];
$content = getValue($content, 'horizontalListRenderer', 'expandedShelfContentsRenderer');
} else {
$sectionTitle = $sectionListRenderer['subMenu']['channelSubMenuRenderer']['contentTypeSubMenuItems'][0]['title'];
$content = $content['gridRenderer'];
}
array_push($itemsArray, [$sectionTitle, $content['items']]);
}
} else {
$result = getContinuationJson($continuationToken);
$itemsArray = [[null, getContinuationItems($result)]];
}
// Note that if there is a `Created playlist`, then there isn't any pagination mechanism on YouTube UI.
// This comment was assuming that they were only `Created playlists` and `Saved playlists`, which isn't the case.
$c4TabbedHeaderRenderer = $result['header']['c4TabbedHeaderRenderer'];
$authorChannelName = $c4TabbedHeaderRenderer['title'];
$authorChannelHandle = $c4TabbedHeaderRenderer['channelHandleText']['runs'][0]['text'];
$authorChannelApproval = $c4TabbedHeaderRenderer['badges'][0]['metadataBadgeRenderer']['tooltip'];
$playlistSections = [];
foreach($itemsArray as [$sectionTitle, $items]) {
// Note that empty playlists aren't listed at all.
$sectionPlaylists = [];
$path = 'continuationItemRenderer/continuationEndpoint/continuationCommand/token';
$lastItem = !empty($items) ? end($items) : [];
if (doesPathExist($lastItem, $path)) {
$nextPageToken = getValue($lastItem, $path);
$items = array_slice($items, 0, count($items) - 1);
}
$isCreatedPlaylists = $sectionTitle === 'Created playlists';
foreach($items as $sectionPlaylistItem) {
if (array_key_exists('showRenderer', $sectionPlaylistItem)) {
continue;
}
$playlistRenderer = getValue($sectionPlaylistItem, 'gridPlaylistRenderer', defaultValue: getValue($sectionPlaylistItem, 'playlistRenderer', 'gridShowRenderer'));
$runs = $playlistRenderer['shortBylineText']['runs'];
if ($isCreatedPlaylists) {
$runs = [null];
}
$authors = !empty($runs) ? array_values(array_filter(array_map(function($shortBylineRun) use ($isCreatedPlaylists, $authorChannelName, $authorChannelHandle, $id, $authorChannelApproval) {
$shortBylineNavigationEndpoint = $shortBylineRun['navigationEndpoint'];
$channelHandle = $shortBylineNavigationEndpoint['commandMetadata']['webCommandMetadata']['url'];
return [
// The following fields `channel*` are `null` without additional code for the `Created playlists` section if there are multiple videos in this section.
'channelName' => $isCreatedPlaylists ? $authorChannelName : $shortBylineRun['text'],
'channelHandle' => $isCreatedPlaylists ? $authorChannelHandle : (str_starts_with($channelHandle, "/@") ? substr($channelHandle, 1) : null),
'channelId' => $isCreatedPlaylists ? $id : $shortBylineNavigationEndpoint['browseEndpoint']['browseId'],
'channelApproval' => $isCreatedPlaylists ? $authorChannelApproval : $playlistRenderer['ownerBadges'][0]['metadataBadgeRenderer']['tooltip'],
];
}, $runs), fn($author) => $author['channelName'] !== ', ')) : [];
$thumbnailRenderer = $playlistRenderer['thumbnailRenderer'];
// For unknown reasons, the playlist `OLAK5uy_ku1ocdOmuBzWb3XrtrAQglseslpye5eIw` has achieved to have a custom thumbnail according to YouTube UI source code.
// The playlist `Playlist with a thumbnail different than the first video one` on https://www.youtube.com/@anothertestagain5569/playlists isn't detected as using a custom thumbnail.
$isThumbnailAVideo = $thumbnailRenderer === null || array_key_exists('playlistVideoThumbnailRenderer', $thumbnailRenderer);
$thumbnailRendererField = 'playlist' . ($isThumbnailAVideo ? 'Video' : 'Custom') . 'ThumbnailRenderer';
if (!array_key_exists($thumbnailRendererField, $thumbnailRenderer)) {
$thumbnailRendererField = 'showCustomThumbnailRenderer';
}
$thumbnailVideo = getVideoFromItsThumbnails($thumbnailRenderer[$thumbnailRendererField]['thumbnail'], $isThumbnailAVideo);
$firstVideos = getFirstVideos($playlistRenderer);
$title = $playlistRenderer['title'];
if (array_key_exists('playlistId', $playlistRenderer)) {
$id = $playlistRenderer['playlistId'];
} else {
$browseId = $playlistRenderer['navigationEndpoint']['browseEndpoint']['browseId'];
// For instance https://www.youtube.com/@FlyMinimal/playlists contains https://www.youtube.com/show/SCG2QET_lsEE-lsp4Kjk7HjA while neither `SCG2QET_lsEE-lsp4Kjk7HjA` nor `G2QET_lsEE-lsp4Kjk7HjA` are correct playlist ids.
// While https://www.youtube.com/@Goldenmoustache/playlists contains https://www.youtube.com/playlist?list=PLHYVKdTa8XHWIEnerFQ-X2dQT5liZpDix which is encoded as `VLPLHYVKdTa8XHWIEnerFQ-X2dQT5liZpDix` in `$browseId` which still seems more appropriate than the `$playlistRenderer['navigationEndpoint']['commandMetadata']['webCommandMetadata']['url']` which contains `/playlist?list=PLHYVKdTa8XHWIEnerFQ-X2dQT5liZpDix`.
// Note that https://www.youtube.com/@FlyMinimal/playlists only contain in its source code `SCG2QET_lsEE-lsp4Kjk7HjA`.
// Only `$browseId` and `url` are common fields to both of these edge cases.
if (str_starts_with($browseId, 'VL')) {
$browseId = substr($browseId, 2);
}
$id = $browseId;
}
$videoCount = intval(getValue($playlistRenderer, 'videoCountText', 'thumbnailOverlays/0/thumbnailOverlayBottomPanelRenderer/text')['runs'][0]['text']);
$sectionPlaylist = [
'id' => $id,
'thumbnailVideo' => $thumbnailVideo,
'firstVideos' => $firstVideos,
'title' => getValue($title, 'runs/0/text', 'simpleText'),
'videoCount' => $videoCount,
'authors' => $authors,
// Does it always start with `Updated `?
// Note that for channels we don't have this field.
'publishedTimeText' => $playlistRenderer['publishedTimeText']['simpleText']
];
array_push($sectionPlaylists, $sectionPlaylist);
}
array_push($playlistSections, [
'title' => $sectionTitle,
'playlists' => $sectionPlaylists,
'nextPageToken' => $nextPageToken
]);
}
$item['playlistSections'] = $playlistSections;
}
if ($options['popular'])
{
$getRendererItems = function($result)
{
$contents = getTabs($result)[0]['tabRenderer']['content']['sectionListRenderer']['contents'];
$shelfRendererPath = 'itemSectionRenderer/contents/0/shelfRenderer';
$content = array_values(array_filter($contents, fn($content) => getValue($content, $shelfRendererPath)['title']['runs'][0]['text'] == 'Popular'))[0];
$shelfRenderer = getValue($content, $shelfRendererPath);
$gridRendererItems = $shelfRenderer['content']['gridRenderer']['items'];
return $gridRendererItems;
};
$item['popular'] = getVideos($item, "https://www.youtube.com/channel/$id", $getRendererItems, $continuationToken);
}
if ($options['recent'])
{
$item['recent'] = getVideos($item, "https://www.youtube.com/channel/$id/recent", fn($result) => getTabByName($result, 'Recent')['tabRenderer']['content']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'][0]['gridRenderer']['items'], $continuationToken);
}
if ($options['letsPlay'])
{
$letsPlay = [];
$result = getJSONFromHTML("https://www.youtube.com/channel/$id/letsplay", forceLanguage: true);
$gridRendererItems = getTabByName($result, 'Let\'s play')['tabRenderer']['content']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'][0]['shelfRenderer']['content']['gridRenderer']['items'];
foreach($gridRendererItems as $gridRendererItem)
{
$gridPlaylistRenderer = $gridRendererItem['gridPlaylistRenderer'];
$titleRun = $gridPlaylistRenderer['title']['runs'][0];
$playlistAuthorRun = $gridPlaylistRenderer['longBylineText']['runs'][0];
$playlistAuthorBrowseEndpoint = $playlistAuthorRun['navigationEndpoint']['browseEndpoint'];
array_push($letsPlay, [
'id' => $gridPlaylistRenderer['playlistId'],
'thumbnails' => $gridPlaylistRenderer['thumbnail']['thumbnails'],
'title' => $titleRun['text'],
'firstVideos' => getFirstVideos($gridPlaylistRenderer),
'videoCount' => intval($gridPlaylistRenderer['videoCountText']['runs'][0]['text']),
'authorName' => $playlistAuthorRun['text'],
'authorChannelId' => $playlistAuthorBrowseEndpoint['browseId'],
'authorChannelHandle' => substr($playlistAuthorBrowseEndpoint['canonicalBaseUrl'], 1),
]);
}
$item['letsPlay'] = $letsPlay;
}
return $item;
}
function returnItems($items)
{
$answer = [
'kind' => 'youtube#channelListResponse',
'etag' => 'NotImplemented',
'items' => $items
];
// should add in some way the pageInfo ?
return json_encode($answer, JSON_PRETTY_PRINT);
}
function getAPI($ids, $order, $continuationToken)
{
$items = [];
foreach ($ids as $id) {
array_push($items, getItem($id, $order, $continuationToken));
}
return returnItems($items);
}
function getVisitorData($result)
{
return $result['responseContext']['webResponseContextExtensionData']['ytConfigData']['visitorData'];
}
function getVideo($gridRendererItem)
{
$gridVideoRenderer = $gridRendererItem['gridVideoRenderer'];
$run = $gridVideoRenderer['shortBylineText']['runs'][0];
$browseEndpoint = $run['navigationEndpoint']['browseEndpoint'];
$title = $gridVideoRenderer['title'];
$publishedAt = getPublishedAt(end(explode('views', $title['accessibility']['accessibilityData']['label'])));
return [
'videoId' => $gridVideoRenderer['videoId'],
'thumbnails' => $gridVideoRenderer['thumbnail']['thumbnails'],
'title' => $title['runs'][0]['text'],
'publishedAt' => $publishedAt,
'views' => getIntFromViewCount($gridVideoRenderer['viewCountText']['simpleText']),
'channelTitle' => $run['text'],
'channelId' => $browseEndpoint['browseId'],
'channelHandle' => substr($browseEndpoint['canonicalBaseUrl'], 1),
'duration' => getIntFromDuration($gridVideoRenderer['thumbnailOverlays'][0]['thumbnailOverlayTimeStatusRenderer']['text']['simpleText']),
'approval' => $gridVideoRenderer['ownerBadges'][0]['metadataBadgeRenderer']['tooltip'],
];
}
function getVideos(&$item, $url, $getGridRendererItems, $continuationToken)
{
$videos = [];
if ($continuationToken === '') {
$result = getJSONFromHTML($url, forceLanguage: true);
$gridRendererItems = $getGridRendererItems($result);
$visitorData = getVisitorData($result);
}
else
{
$result = getContinuationJson($continuationToken);
$gridRendererItems = getContinuationItems($result);
}
foreach($gridRendererItems as $gridRendererItem)
{
if(!array_key_exists('continuationItemRenderer', $gridRendererItem))
{
array_push($videos, getVideo($gridRendererItem));
}
}
if($gridRendererItem != null && array_key_exists('continuationItemRenderer', $gridRendererItem))
{
$item['nextPageToken'] = $gridRendererItem['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'] . ',' . $visitorData;
}
return $videos;
}
function getVideoFromItsThumbnails($videoThumbnails, $isVideo = true) {
$videoThumbnails = $videoThumbnails['thumbnails'];
// Maybe we can simplify URLs, as we used to as follows, but keep in mind that the resolution or the access may then become incorrect for both kind of thumbnails.
//$videoThumbnails[0]['url'] = explode('?', $videoThumbnails[0]['url'])[0];
$videoId = $isVideo ? substr($videoThumbnails[0]['url'], 23, 11) : null;
return [
'id' => $videoId,
'thumbnails' => $videoThumbnails
];
}
function getFirstVideos($playlistRenderer)
{
$firstVideos = array_key_exists('thumbnail', $playlistRenderer) ? [getVideoFromItsThumbnails($playlistRenderer['thumbnail'])] : array_map(fn($videoThumbnails) => getVideoFromItsThumbnails($videoThumbnails), getValue($playlistRenderer, 'thumbnails', defaultValue: []));
$sidebarThumbnails = $playlistRenderer['sidebarThumbnails'];
$secondToFourthVideo = $sidebarThumbnails !== null ? array_map(fn($videoThumbnails) => getVideoFromItsThumbnails($videoThumbnails), $sidebarThumbnails) : [];
$firstVideos = array_merge($firstVideos, $secondToFourthVideo);
return $firstVideos;
}
================================================
FILE: commentThreads.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
// Stack Overflow source: https://stackoverflow.com/q/71186488
// use multiple lines, or not as they are not supposed to change
$commentThreadsTests = [
// How to disable people comments?
// Otherwise should have a private set of tests
//['part=snippet&videoId=UC4QobU6STFB0P71PMvOGN5A&order=viewCount', 'items/0/id/videoId', 'jNQXAC9IVRw'],
//['part=snippet,replies&commentId=UgzT9BA9uQhXw05Q2Ip4AaABAg&videoId=mWdFMNQBcjs', 'items/0/id/videoId', 'jNQXAC9IVRw'],
];
// example: https://youtu.be/mrJachWLjHU
// example: https://youtu.be/DyDfgMOUjCI
include_once 'common.php';
$realOptions = [
'snippet',
'replies',
];
foreach ($realOptions as $realOption) {
$options[$realOption] = false;
}
if (isset($_GET['part'])) {
$part = $_GET['part'];
$parts = explode(',', $part, count($realOptions));
foreach ($parts as $part) {
if (!in_array($part, $realOptions)) {
dieWithJsonMessage("Invalid part $part");
} else {
$options[$part] = true;
}
}
$videoId = null;
if (isset($_GET['videoId'])) {
$videoId = $_GET['videoId'];
if (!isVideoId($videoId)) {
dieWithJsonMessage('Invalid videoId');
}
}
$commentId = null;
if (isset($_GET['id'])) {
$commentId = $_GET['id'];
if (!isCommentId($commentId)) {
dieWithJsonMessage('Invalid id');
}
}
$order = isset($_GET['order']) ? $_GET['order'] : 'relevance';
if (!in_array($order, ['relevance', 'time'])) {
dieWithJsonMessage('Invalid order');
}
$continuationToken = '';
if (isset($_GET['pageToken'])) {
$continuationToken = $_GET['pageToken'];
if (!isContinuationToken($continuationToken)) {
dieWithJsonMessage('Invalid pageToken');
}
}
echo getAPI($videoId, $commentId, $order, $continuationToken);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function getAPI($videoId, $commentId, $order, $continuationToken, $simulatedContinuation = false)
{
if($commentId !== null)
{
$result = getJSONFromHTML("https://www.youtube.com/watch?v=$videoId&lc=$commentId");
$continuationToken = $result['contents']['twoColumnWatchNextResults']['results']['results']['contents'][3]['itemSectionRenderer']['contents'][0]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'];
}
$continuationTokenProvided = $continuationToken != '';
if ($continuationTokenProvided) {
$rawData = [
'context' => [
'client' => [
'clientName' => 'WEB',
'clientVersion' => MUSIC_VERSION
]
],
'continuation' => $continuationToken
];
$opts = [
'http' => [
'method' => 'POST',
'header' => ['Content-Type: application/json'],
'content' => json_encode($rawData),
]
];
$result = getJSON('https://www.youtube.com/youtubei/v1/' . ($videoId !== null ? 'next' : 'browse') . '?key=' . UI_KEY, $opts);
if ($order === 'time' && $simulatedContinuation) {
$continuationToken = $result['onResponseReceivedEndpoints'][0]['reloadContinuationItemsCommand']['continuationItems'][0]['commentsHeaderRenderer']['sortMenu']['sortFilterSubMenuRenderer']['subMenuItems'][1]['serviceEndpoint']['continuationCommand']['token'];
return getAPI($videoId, $commentId, null, $continuationToken);
}
} else {
$result = getJSONFromHTML("https://www.youtube.com/watch?v=$videoId");
$continuationToken = end($result['contents']['twoColumnWatchNextResults']['results']['results']['contents'])['itemSectionRenderer']['contents'][0]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'];
if($continuationToken != '') {
return getAPI($videoId, $commentId, $order, $continuationToken, true);
}
}
$answerItems = [];
$items = $result['frameworkUpdates']['entityBatchUpdate']['mutations'];
$isTopLevelComment = true;
foreach ($items as $item) {
$payload = $item['payload'];
if (array_key_exists('engagementToolbarStateEntityPayload', $payload)) {
$answerItems[$item['entityKey']]['snippet']['topLevelComment']['snippet']['creatorHeart'] = $payload['engagementToolbarStateEntityPayload']['heartState'] == 'TOOLBAR_HEART_STATE_HEARTED';
}
if (!array_key_exists('commentEntityPayload', $payload)) {
continue;
}
$comment = $payload['commentEntityPayload'];
$properties = $comment['properties'];
$author = $comment['author'];
$toolbar = $comment['toolbar'];
$publishedAt = $properties['publishedTime'];
$publishedAt = str_replace(' (edited)', '', $publishedAt, $count);
$internalSnippet = [
'content' => $properties['content']['content'],
'publishedAt' => $publishedAt,
'wasEdited' => $count > 0,
'authorChannelId' => $author['channelId'],
'authorHandle' => $author['displayName'],
'authorName' => str_replace('❤ by ', '', $toolbar['heartActiveTooltip']),
'authorAvatar' => $comment['avatar']['image']['sources'][0],
'isCreator' => $author['isCreator'],
'isArtist' => $author['isArtist'],
'likeCount' => getIntValue($toolbar['likeCountLiked']),
'totalReplyCount' => intval($toolbar['replyCount']),
'videoCreatorHasReplied' => false,
'isPinned' => false,
];
$commentId = $properties['commentId'];
$answerItem = [
'kind' => 'youtube#comment' . ($isTopLevelComment ? 'Thread' : ''),
'etag' => 'NotImplemented',
'id' => $commentId,
'snippet' => ($isTopLevelComment ? [
'topLevelComment' => [
'kind' => 'youtube#comment',
'etag' => 'NotImplemented',
'id' => $commentId,
'snippet' => $internalSnippet
]
] : $internalSnippet)
];
$answerItems[$properties['toolbarStateKey']] = $answerItem;
}
$continuationItems = $result['onResponseReceivedEndpoints'][1]['reloadContinuationItemsCommand']['continuationItems'];
foreach ($continuationItems as $item) {
$commentThreadRenderer = $item['commentThreadRenderer'];
$toolbarStateKey = $commentThreadRenderer['commentViewModel']['commentViewModel']['toolbarStateKey'];
// How to avoid repeating path?
if (doesPathExist($commentThreadRenderer, 'replies/commentRepliesRenderer/viewRepliesCreatorThumbnail')) {
$answerItems[$toolbarStateKey]['snippet']['topLevelComment']['snippet']['videoCreatorHasReplied'] = true;
}
if (doesPathExist($commentThreadRenderer, 'commentViewModel/commentViewModel/pinnedText')) {
$answerItems[$toolbarStateKey]['snippet']['topLevelComment']['snippet']['isPinned'] = true;
}
if ($toolbarStateKey !== null) {
$answerItems[$toolbarStateKey]['snippet']['topLevelComment']['snippet']['nextPageToken'] = $commentThreadRenderer['replies']['commentRepliesRenderer']['contents'][0]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'];
}
}
$answerItems = array_values($answerItems);
$nextContinuationToken = $continuationItems[20]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'];
$answer = [
'kind' => 'youtube#comment' . ($isTopLevelComment ? 'Thread' : '') . 'ListResponse',
'etag' => 'NotImplemented',
'pageInfo' => [
'totalResults' => intval($result['onResponseReceivedEndpoints'][0]['reloadContinuationItemsCommand']['continuationItems'][0]['commentsHeaderRenderer']['countText']['runs'][0]['text']),
'resultsPerPage' => $isTopLevelComment ? 20 : 10
]
];
if ($nextContinuationToken != '') {
$answer['nextPageToken'] = $nextContinuationToken;
}
$answer['items'] = $answerItems;
return json_encode($answer, JSON_PRETTY_PRINT);
}
================================================
FILE: common.php
================================================
<?php
include_once 'constants.php';
include_once 'configuration.php';
ini_set('display_errors', 0);
if(RESTRICT_USAGE_TO_KEY !== '')
{
if(isset($_GET['instanceKey']))
{
if($_GET['instanceKey'] !== RESTRICT_USAGE_TO_KEY)
{
die("The provided <code>instanceKey</code> isn't correct!");
}
}
else
{
die('This instance requires that you provide the appropriate <code>instanceKey</code> parameter!');
}
}
function getContextFromOpts($opts)
{
if (GOOGLE_ABUSE_EXEMPTION !== '') {
// Can maybe leverage an approach like [issues/321](https://github.com/Benjamin-Loison/YouTube-operational-API/issues/321).
$cookieToAdd = 'GOOGLE_ABUSE_EXEMPTION=' . GOOGLE_ABUSE_EXEMPTION;
// Can't we simplify the following code?
if (array_key_exists('http', $opts)) {
$http = $opts['http'];
if (array_key_exists('header', $http)) {
$headers = $http['header'];
$isThereACookieHeader = false;
foreach ($headers as $headerIndex => $header) {
if (str_starts_with($header, 'Cookie: ')) {
$opts['http']['header'][$headerIndex] = "$header; $cookieToAdd";
$isThereACookieHeader = true;
break;
}
}
if (!$isThereACookieHeader) {
array_push($opts['http']['header'], "Cookie: $cookieToAdd");
}
}
} else {
$opts = [
'http' => [
'header' => [
"Cookie: $cookieToAdd"
]
]
];
}
}
$context = stream_context_create($opts);
return $context;
}
function getHeadersFromOpts($url, $opts)
{
$context = getContextFromOpts($opts);
$headers = get_headers($url, true, $context);
return $headers;
}
function fileGetContentsAndHeadersFromOpts($url, $opts)
{
if(HTTPS_PROXY_ADDRESS !== '')
{
if(!array_key_exists('http', $opts))
{
$opts['http'] = [];
}
$opts['http']['proxy'] = 'tcp://' . HTTPS_PROXY_ADDRESS . ':' . HTTPS_PROXY_PORT;
$opts['http']['request_fulluri'] = true;
if(HTTPS_PROXY_USERNAME !== '')
{
$headers = getValue($opts['http'], 'header', $defaultValue = []);
array_push($headers, 'Proxy-Authorization: Basic ' . base64_encode(HTTPS_PROXY_USERNAME . ':' . HTTPS_PROXY_PASSWORD));
$opts['http']['header'] = $headers;
}
}
$context = getContextFromOpts($opts);
$result = file_get_contents($url, false, $context);
return [$result, $http_response_header];
}
function isRedirection($url)
{
$opts = [
'http' => [
'ignore_errors' => true,
'follow_location' => false,
]
];
$http_response_header = getHeadersFromOpts($url, $opts);
$code = intval(explode(' ', $http_response_header[0])[1]);
if (in_array($code, HTTP_CODES_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC)) {
detectedAsSendingUnusualTraffic();
}
return $code == 303;
}
function getRemote($url, $opts = [], $verifyTrafficIfForbidden = true)
{
[$result, $headers] = fileGetContentsAndHeadersFromOpts($url, $opts);
foreach (HTTP_CODES_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC as $HTTP_CODE_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC) {
if (str_contains($headers[0], strval($HTTP_CODE_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC)) && ($HTTP_CODE_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC != 403 || $verifyTrafficIfForbidden)) {
detectedAsSendingUnusualTraffic();
}
}
return $result;
}
function dieWithJsonMessage($message, $code = 400)
{
$error = [
'code' => $code,
'message' => $message
];
$result = [
'error' => $error
];
die(json_encode($result, JSON_PRETTY_PRINT));
}
function detectedAsSendingUnusualTraffic()
{
dieWithJsonMessage('YouTube has detected unusual traffic from this YouTube operational API instance. Please try your request again later or see alternatives at https://github.com/Benjamin-Loison/YouTube-operational-API/issues/11', 403);
}
function getJSON($url, $opts = [], $verifyTrafficIfForbidden = true)
{
return json_decode(getRemote($url, $opts, $verifyTrafficIfForbidden), true);
}
function getJSONStringFromHTMLScriptPrefix($html, $scriptPrefix)
{
$html = explode(';</script>', explode("\">$scriptPrefix", $html, 3)[1], 2)[0];
return $html;
}
function getJSONFromHTMLScriptPrefix($html, $scriptPrefix)
{
$html = getJSONStringFromHTMLScriptPrefix($html, $scriptPrefix);
return json_decode($html, true);
}
function getJSONStringFromHTML($html, $scriptVariable = '', $prefix = 'var ')
{
// don't use as default variable because getJSONFromHTML call this function with empty string
if ($scriptVariable === '') {
$scriptVariable = 'ytInitialData';
}
return getJSONStringFromHTMLScriptPrefix($html, "$prefix$scriptVariable = ");
}
function getJSONFromHTML($url, $opts = [], $scriptVariable = '', $prefix = 'var ', $forceLanguage = false, $verifiesChannelRedirection = false)
{
if($forceLanguage) {
$HEADER = 'Accept-Language: en';
if(!doesPathExist($opts, 'http/header')) {
$opts['http']['header'] = [$HEADER];
} else {
array_push($opts['http']['header'], $HEADER);
}
}
$html = getRemote($url, $opts);
$jsonStr = getJSONStringFromHTML($html, $scriptVariable, $prefix);
$json = json_decode($jsonStr, true);
if($verifiesChannelRedirection)
{
$redirectedToChannelIdPath = 'onResponseReceivedActions/0/navigateAction/endpoint/browseEndpoint/browseId';
if(doesPathExist($json, $redirectedToChannelIdPath))
{
$redirectedToChannelId = getValue($json, $redirectedToChannelIdPath);
$url = preg_replace('/[\w\-_]{24}/', $redirectedToChannelId, $url);
// Does a redirection of redirection for a channel exist?
return getJSONFromHTML($url, $opts, $scriptVariable, $prefix, $forceLanguage, $verifiesChannelRedirection);
}
}
return $json;
}
function checkRegex($regex, $str)
{
return preg_match("/^$regex$/", $str) === 1;
}
function isContinuationToken($continuationToken)
{
return checkRegex('[\w=\-_]+', $continuationToken);
}
function isContinuationTokenAndVisitorData($continuationTokenAndVisitorData)
{
return checkRegex('[\w=\-_]+,[\w=\-_]*', $continuationTokenAndVisitorData);
}
function isPlaylistId($playlistId)
{
return checkRegex('[\w\-_]+', $playlistId);
}
// What's the minimal length ?
// Are there forbidden characters?
function isCId($cId)
{
return true;
}
function isUsername($username)
{
return checkRegex('\w+', $username);
}
function isChannelId($channelId)
{
return checkRegex('UC[\w\-_]{22}', $channelId);
}
function isVideoId($videoId)
{
return checkRegex('[\w\-_]{11}', $videoId);
}
function isHashtag($hashtag)
{
return true; // checkRegex('[\w_]+', $hashtag); // 'é' is a valid hashtag for instance
}
function isSAPISIDHASH($SAPISIDHASH)
{
return checkRegex('[1-9]\d{9}_[a-f\d]{40}', $SAPISIDHASH);
}
function isQuery($q)
{
return true; // should restrain
}
function isClipId($clipId)
{
return checkRegex('Ug[\w\-_]{34}', $clipId);
}
function isEventType($eventType)
{
return in_array($eventType, ['completed', 'live', 'upcoming']);
}
function isPositiveInteger($s)
{
return preg_match("/^\d+$/", $s);
}
function isYouTubeDataAPIV3Key($youtubeDataAPIV3Key)
{
return checkRegex('AIzaSy[A-D][\w\-_]{32}', $youtubeDataAPIV3Key);
}
function isHandle($handle)
{
return checkRegex('@[\w\-_.]{3,}', $handle);
}
function isPostId($postId)
{
return (checkRegex('Ug[w-z][\w\-_]{16}4AaABCQ', $postId) || checkRegex('Ugkx[\w\-_]{32}', $postId));
}
function isCommentId($commentId)
{
return checkRegex('Ug[w-z][\w\-_]{16}4AaABAg(|.[\w\-]{22})', $commentId);
}
// Assume `$path !== ''`.
function doesPathExist($json, $path)
{
if ($json === null) {
return false;
}
$parts = explode('/', $path);
$partsCount = count($parts);
if ($partsCount == 1) {
return array_key_exists($path, $json);
}
return array_key_exists($parts[0], $json) && doesPathExist($json[$parts[0]], join('/', array_slice($parts, 1, $partsCount - 1)));
}
function getValue($json, $path, $defaultPath = null, $defaultValue = null)
{
// Alternatively could make a distinct return for `getValue` depending on path found or not to avoid `null` ambiguity.
if(!doesPathExist($json, $path))
{
return $defaultPath !== null ? getValue($json, $defaultPath) : $defaultValue;
}
$parts = explode('/', $path);
$partsCount = count($parts);
if ($partsCount == 1) {
return $json[$path];
}
$value = getValue($json[$parts[0]], join('/', array_slice($parts, 1, $partsCount - 1)));
return $value;
}
function getIntValue($unitCount, $unit = '')
{
$unitCount = str_replace(',', '', $unitCount);
$unitCount = str_replace(" {$unit}s", '', $unitCount);
$unitCount = str_replace(" $unit", '', $unitCount);
if($unitCount === 'No') {
$unitCount = '0';
}
$unitCount = str_replace('K', '*1_000', $unitCount);
$unitCount = str_replace('M', '*1_000_000', $unitCount);
$unitCount = str_replace('B', '*1_000_000_000', $unitCount);
if(checkRegex('[\d_.*KMB]+', $unitCount)) {
$unitCount = eval("return round($unitCount);");
}
return intval($unitCount);
}
function getCommunityPostFromContent($content)
{
$backstagePost = $content['backstagePostThreadRenderer']['post']; // for posts that are shared from other channels
$common = getValue($backstagePost, 'backstagePostRenderer', 'sharedPostRenderer');
$id = $common['postId'];
$channelId = $common['authorEndpoint']['browseEndpoint']['browseId'];
// Except for `Image`, all other posts require text.
$contentText = [];
$textContent = getValue($common, 'contentText', 'content'); // sharedPosts have the same content just in slightly different positioning
foreach ($textContent['runs'] as $textCommon) {
$contentTextItem = ['text' => $textCommon['text']];
if (array_key_exists('navigationEndpoint', $textCommon)) {
// `$url` isn't defined.
if (str_starts_with($url, 'https://www.youtube.com/redirect?')) {
// `$text` isn't defined here.
$contentTextItem['url'] = $text;
} else {
$navigationEndpoint = $textCommon['navigationEndpoint'];
$url = getValue($navigationEndpoint, 'commandMetadata/webCommandMetadata/url', 'browseEndpoint/canonicalBaseUrl');
$contentTextItem['url'] = "https://www.youtube.com$url";
}
}
array_push($contentText, $contentTextItem);
}
$backstageAttachment = [];
if (array_key_exists('backstageAttachment', $common)) {
$backstageAttachment = $common['backstageAttachment'];
}
$images = [];
if (array_key_exists('backstageImageRenderer', $backstageAttachment)) {
$images = [$backstageAttachment['backstageImageRenderer']['image']];
} else if (array_key_exists('postMultiImageRenderer', $backstageAttachment)) {
foreach($backstageAttachment['postMultiImageRenderer']['images'] as $image) {
array_push($images, $image['backstageImageRenderer']['image']);
}
}
$videoId = getValue($backstageAttachment, 'videoRenderer/videoId');
$date = $common['publishedTimeText']['runs'][0]['text'];
$edited = str_ends_with($date, ' (edited)');
$date = str_replace(' (edited)', '', $date);
$date = str_replace('shared ', '', $date);
$sharedPostId = getValue($common, 'originalPost/backstagePostRenderer/postId');
$poll = null;
if (array_key_exists('pollRenderer', $backstageAttachment)) {
$pollRenderer = $backstageAttachment['pollRenderer'];
$choices = [];
foreach ($pollRenderer['choices'] as $choice) {
$returnedChoice = $choice['text']['runs'][0];
$returnedChoice['image'] = $choice['image'];
$returnedChoice['voteRatio'] = $choice['voteRatioIfNotSelected'];
array_push($choices, $returnedChoice);
}
$totalVotesStr = $pollRenderer['totalVotes']['simpleText'];
// What if no vote? Note that haven't seen a poll with a single vote.
$totalVotes = intval(str_replace(' vote', '', str_replace(' votes', '', $totalVotesStr)));
$poll = [
'choices' => $choices,
'totalVotes' => $totalVotes
];
}
$likes = getIntValue(getValue($common, 'voteCount/simpleText', defaultValue : 0));
// Retrieving comments when using `community?part=snippet` requires another HTTPS request to `browse` YouTube UI endpoint.
// sharedPosts do not have 'actionButtons' so this next line will end up defaulting to 0 $comments
$commentsPath = 'actionButtons/commentActionButtonsRenderer/replyButton/buttonRenderer';
$commentsCommon = doesPathExist($common, $commentsPath) ? getValue($common, $commentsPath) : $common;
$post = [
'id' => $id,
'channelId' => $channelId,
'channelName' => $common['authorText']['runs'][0]['text'],
'channelHandle' => substr($common['authorEndpoint']['browseEndpoint']['canonicalBaseUrl'], 1),
'channelThumbnails' => array_map(function($thumbnail) { $thumbnail['url'] = 'https:' . $thumbnail['url']; return $thumbnail; }, $common['authorThumbnail']['thumbnails']),
'date' => $date,
'contentText' => $contentText,
'likes' => $likes,
'videoId' => $videoId,
'images' => $images,
'poll' => $poll,
'edited' => $edited,
'sharedPostId' => $sharedPostId,
];
if(array_key_exists('text', $commentsCommon))
{
$commentsCount = getIntValue($commentsCommon['text']['simpleText']);
$post['commentsCount'] = $commentsCount;
}
return $post;
}
function getIntFromViewCount($viewCount)
{
if ($viewCount === 'No views') {
$viewCount = 0;
} else {
foreach([',', ' views', 'view'] as $toRemove) {
$viewCount = str_replace($toRemove, '', $viewCount);
}
} // don't know if the 1 view case is useful
$viewCount = intval($viewCount);
return $viewCount;
}
function getIntFromDuration($timeStr)
{
$isNegative = $timeStr[0] === '-';
if ($isNegative) {
$timeStr = substr($timeStr, 1);
}
$format = 'j:H:i:s';
$timeParts = explode(':', $timeStr);
$timePartsCount = count($timeParts);
$minutes = $timeParts[$timePartsCount - 2];
$timeParts[$timePartsCount - 2] = strlen($minutes) == 1 ? "0$minutes" : $minutes;
$timeStr = implode(':', $timeParts);
for ($timePartsIndex = 0; $timePartsIndex < 4 - $timePartsCount; $timePartsIndex++) {
$timeStr = "00:$timeStr";
}
while (date_parse_from_format($format, $timeStr) === false) {
$format = substr($format, 2);
}
$timeComponents = date_parse_from_format($format, $timeStr);
$timeInt = $timeComponents['day'] * (3600 * 24) +
$timeComponents['hour'] * 3600 +
$timeComponents['minute'] * 60 +
$timeComponents['second'];
return ($isNegative ? -1 : 1) * $timeInt;
}
function getFirstNodeContainingPath($nodes, $path) {
return array_values(array_filter($nodes, fn($node) => doesPathExist($node, $path)))[0];
}
function getTabByName($result, $tabName) {
if (array_key_exists('contents', $result)) {
return array_values(array_filter(getTabs($result), fn($tab) => (getValue($tab, 'tabRenderer/title') === $tabName)))[0];
} else {
return null;
}
}
function getPublishedAt($publishedAtRaw) {
$publishedAtStr = str_replace('ago', '', $publishedAtRaw);
$publishedAtStr = str_replace('seconds', '* 1 +', $publishedAtStr);
$publishedAtStr = str_replace('second', '* 1 +', $publishedAtStr);
$publishedAtStr = str_replace('minutes', '* 60 +', $publishedAtStr);
$publishedAtStr = str_replace('minute', '* 60 +', $publishedAtStr);
$publishedAtStr = str_replace('hours', '* 3600 +', $publishedAtStr);
$publishedAtStr = str_replace('hour', '* 3600 +', $publishedAtStr);
$publishedAtStr = str_replace('days', '* 86400 +', $publishedAtStr);
$publishedAtStr = str_replace('day', '* 86400 +', $publishedAtStr);
$publishedAtStr = str_replace('weeks', '* 604800 +', $publishedAtStr);
$publishedAtStr = str_replace('week', '* 604800 +', $publishedAtStr);
$publishedAtStr = str_replace('months', '* 2592000 +', $publishedAtStr); // not sure
$publishedAtStr = str_replace('month', '* 2592000 +', $publishedAtStr);
$publishedAtStr = str_replace('years', '* 31104000 +', $publishedAtStr); // not sure
$publishedAtStr = str_replace('year', '* 31104000 +', $publishedAtStr);
// To remove last ` +`.
$publishedAtStr = substr($publishedAtStr, 0, strlen($publishedAtStr) - 2);
$publishedAtStr = str_replace(' ', '', $publishedAtStr); // "security"
$publishedAtStr = str_replace(',', '', $publishedAtStr);
$publishedAtStrLen = strlen($publishedAtStr);
// "security"
for ($publishedAtStrIndex = $publishedAtStrLen - 1; $publishedAtStrIndex >= 0; $publishedAtStrIndex--) {
$publishedAtChar = $publishedAtStr[$publishedAtStrIndex];
if (!str_contains('+*0123456789', $publishedAtChar)) {
$publishedAtStr = substr($publishedAtStr, $publishedAtStrIndex + 1, $publishedAtStrLen - $publishedAtStrIndex - 1);
break;
}
}
$publishedAt = time() - eval("return $publishedAtStr;");
// the time is not perfectly accurate this way
return $publishedAt;
}
function test()
{
global $test;
return isset($test);
}
function getContinuationItems($result)
{
return $result['onResponseReceivedActions'][0]['appendContinuationItemsAction']['continuationItems'];
}
function getTabs($result)
{
return $result['contents']['twoColumnBrowseResultsRenderer']['tabs'];
}
function getContinuationJson($continuationToken)
{
$containsVisitorData = str_contains($continuationToken, ',');
if($containsVisitorData)
{
$continuationTokenParts = explode(',', $continuationToken);
$continuationToken = $continuationTokenParts[0];
}
$rawData = [
'context' => [
'client' => [
'clientName' => 'WEB',
'clientVersion' => MUSIC_VERSION
]
],
'continuation' => $continuationToken
];
if($containsVisitorData)
{
$rawData['context']['client']['visitorData'] = $continuationTokenParts[1];
}
$http = [
'header' => [
'Content-Type: application/json'
],
'method' => 'POST',
'content' => json_encode($rawData)
];
$httpOptions = [
'http' => $http
];
$result = getJSON('https://www.youtube.com/youtubei/v1/browse?key=' . UI_KEY, $httpOptions);
return $result;
}
function verifyMultipleIdsConfiguration($realIds, $field) {
if (count($realIds) >= 2 && !MULTIPLE_IDS_ENABLED) {
dieWithJsonMessage("Multiple {$field}s are disabled on this instance");
}
}
function verifyTooManyIds($realIds, $field) {
if (count($realIds) > MULTIPLE_IDS_MAXIMUM) {
dieWithJsonMessage("Too many $field");
}
}
function verifyMultipleIds($realIds, $field = 'id') {
verifyMultipleIdsConfiguration($realIds, $field);
verifyTooManyIds($realIds, $field);
}
function getMultipleIds($field) {
$realIdsString = $_GET[$field];
$realIds = explode(',', $realIdsString);
verifyMultipleIds($realIds);
return $realIds;
}
function includeOnceProto($proto) {
$COMMON_PATH = 'proto/php';
include_once "$COMMON_PATH/$proto.php";
include_once "$COMMON_PATH/GPBMetadata/$proto.php";
}
function includeOnceProtos($protos) {
require_once __DIR__ . '/vendor/autoload.php';
foreach($protos as $proto) {
includeOnceProto($proto);
}
}
// Source: https://www.php.net/manual/en/function.base64-encode.php#103849
function base64url_encode($data) {
return rtrim(strtr(base64_encode($data), '+/', '-_'), '=');
}
?>
================================================
FILE: community.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
include_once 'common.php';
includeOnceProtos(['Browse', 'SubBrowse']);
$realOptions = [
'snippet',
];
foreach ($realOptions as $realOption) {
$options[$realOption] = false;
}
if (isset($_GET['part'], $_GET['id'], $_GET['channelId'])) {
$part = $_GET['part'];
$parts = explode(',', $part, count($realOptions));
foreach ($parts as $part) {
if (!in_array($part, $realOptions)) {
dieWithJsonMessage("Invalid part $part");
} else {
$options[$part] = true;
}
}
$postId = $_GET['id'];
if (!isPostId($postId)) {
dieWithJsonMessage('Invalid postId');
}
$channelId = $_GET['channelId'];
if (!isChannelId($channelId)) {
dieWithJsonMessage('Invalid channelId');
}
$order = isset($_GET['order']) ? $_GET['order'] : 'relevance';
if (!in_array($order, ['relevance', 'time'])) {
dieWithJsonMessage('Invalid order');
}
echo getAPI($postId, $channelId, $order);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function implodeArray($anArray, $separator)
{
return array_map(fn($k, $v) => "${k}${separator}${v}", array_keys($anArray), array_values($anArray));
}
function getAPI($postId, $channelId, $order)
{
$currentTime = time();
$SAPISID = 'CENSORED';
$__Secure_3PSID = 'CENSORED';
$ORIGIN = 'https://www.youtube.com';
$SAPISIDHASH = "${currentTime}_" . sha1("$currentTime $SAPISID $ORIGIN");
$subBrowse = new \SubBrowse();
$subBrowse->setPostId($postId);
$browse = new \Browse();
$browse->setEndpoint('community');
$browse->setSubBrowse($subBrowse);
$params = base64_encode($browse->serializeToString());
$rawData = [
'context' => [
'client' => [
'clientName' => 'WEB',
'clientVersion' => MUSIC_VERSION
]
],
'browseId' => $channelId,
'params' => $params,
];
$opts = [
'http' => [
'method' => 'POST',
'header' => implodeArray([
'Content-Type' => 'application/json',
'Origin' => $ORIGIN,
'Authorization' => "SAPISIDHASH $SAPISIDHASH",
'Cookie' => implode('; ', implodeArray([
'__Secure-3PSID' => $__Secure_3PSID,
'__Secure-3PAPISID' => $SAPISID,
], '=')),
], ': '),
'content' => json_encode($rawData),
]
];
$result = getJSON('https://www.youtube.com/youtubei/v1/browse', $opts);
$contents = getTabByName($result, 'Community')['tabRenderer']['content']['sectionListRenderer']['contents'];
$content = $contents[0]['itemSectionRenderer']['contents'][0];
$post = getCommunityPostFromContent($content);
$continuationToken = urldecode($contents[1]['itemSectionRenderer']['contents'][0]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token']);
if ($order === 'time') {
$result = getContinuationJson($continuationToken);
$continuationToken = urldecode($result['onResponseReceivedEndpoints'][0]['reloadContinuationItemsCommand']['continuationItems'][0]['commentsHeaderRenderer']['sortMenu']['sortFilterSubMenuRenderer']['subMenuItems'][1]['serviceEndpoint']['continuationCommand']['token']);
}
$comments = [
'nextPageToken' => $continuationToken
];
$post['comments'] = $comments;
$answerItem = [
'kind' => 'youtube#community',
'etag' => 'NotImplemented',
'id' => $postId,
'snippet' => $post
];
$answer = [
'kind' => 'youtube#communityListResponse',
'etag' => 'NotImplemented'
];
$answer['items'] = [$answerItem];
return json_encode($answer, JSON_PRETTY_PRINT);
}
================================================
FILE: configuration.php
================================================
<?php
// Global:
define('SERVER_NAME', 'my instance');
// Web-scraping endpoints:
define('GOOGLE_ABUSE_EXEMPTION', '');
define('MULTIPLE_IDS_ENABLED', True);
define('MULTIPLE_IDS_MAXIMUM', 50);
define('HTTPS_PROXY_ADDRESS', '');
define('HTTPS_PROXY_PORT', 80);
define('HTTPS_PROXY_USERNAME', '');
define('HTTPS_PROXY_PASSWORD', '');
// No-key endpoint:
define('KEYS_FILE', 'ytPrivate/keys.txt');
// Both following entries can be generated with `tr -dc A-Za-z0-9 </dev/urandom | head -c 32 ; echo`.
define('RESTRICT_USAGE_TO_KEY', '');
// If not defined, a random value will be used to prevent denial-of-service.
define('ADD_KEY_FORCE_SECRET', '');
define('ADD_KEY_TO_INSTANCES', []);
?>
================================================
FILE: constants.php
================================================
<?php
include_once 'configuration.php';
$newAddKeyForceSecret = ADD_KEY_FORCE_SECRET;
if (ADD_KEY_FORCE_SECRET === '') {
$newAddKeyForceSecret = bin2hex(random_bytes(16));
}
define('NEW_ADD_KEY_FORCE_SECRET', $newAddKeyForceSecret);
use const NEW_ADD_KEY_FORCE_SECRET as ADD_KEY_FORCE_SECRET;
$protocol = (!empty($_SERVER['HTTPS']) && (strtolower($_SERVER['HTTPS']) == 'on')) ? 'https' : 'http';
define('WEBSITE_URL_BASE', "$protocol://{$_SERVER['HTTP_HOST']}");
define('WEBSITE_URL', WEBSITE_URL_BASE . "{$_SERVER['REQUEST_URI']}");
define('SUB_VERSION_STR', '.9999099');
define('MUSIC_VERSION', '2' . SUB_VERSION_STR);
define('CLIENT_VERSION', '1' . SUB_VERSION_STR);
define('UI_KEY', 'AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8'); // this isn't a YouTube Data API v3 key
define('USER_AGENT', 'Firefox/100');
define('HTTP_CODES_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC', [302, 403, 429]);
?>
================================================
FILE: docker-compose.yml
================================================
version: '3.8'
services:
api:
image: youtube-operational-api
build: .
restart: on-failure
ports:
- ${EXPOSED_HTTP_PORT}:80
================================================
FILE: index.php
================================================
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>YouTube operational API</title>
<style>
body {
max-width: 73%;
margin: 5% auto;
font-family: sans-serif;
color: #444;
padding: 0;
}
h1,
h2,
h3 {
line-height: 1.2;
}
p {
word-break: break-word;
}
@media (prefers-color-scheme: dark) {
body {
color: #c9d1d9;
background: #0d1117;
}
a:link {
color: #58a6ff;
}
a:visited {
color: #8e96f0;
}
}
</style>
</head>
<body>
<?php
include_once 'common.php';
function url($url, $name = '')
{
if ($name === '') {
$name = $url;
}
return "<a href=\"$url\">$name</a>";
}
function yt()
{
echo '<a href="https://developers.google.com/youtube/v3">YouTube Data API v3</a>';
}
function getUrl($parameters)
{
return urldecode(http_build_query(array_combine(array_keys($parameters), array_map(fn($parameterValue) => gettype($parameterValue) === 'string' ? $parameterValue : implode(',', $parameterValue), array_values($parameters)))));
}
function feature($feature)
{
$suburl = "$feature[0]/list";
$webpage = explode('/', $suburl, 2)[0];
$url = getUrl($feature[1]) . (count($feature) >= 3 ? '(&' . getUrl($feature[2]) . ')' : '');
$name = ucfirst(str_replace('/', ': ', $suburl));
echo "<p>Based on <a href=\"https://developers.google.com/youtube/v3/docs/$suburl\">$name</a>: " . url(WEBSITE_URL . "$webpage?$url") . '</p>';
}
$features = [
[
'channels',
[
'part' => [
'status',
'upcomingEvents',
'shorts',
'community',
'channels',
'about',
'approval',
'playlists',
'snippet',
'membership',
'popular',
'recent',
'letsPlay',
],
'cId' => 'C_ID',
'id' => 'CHANNEL_ID',
'handle' => 'HANDLE',
'forUsername' => 'USERNAME',
'raw' => 'RAW',
'order' => 'viewCount',
],
[
'pageToken' => 'PAGE_TOKEN',
],
],
[
'commentThreads',
[
'part' => [
'snippet',
'replies',
],
'id' => 'COMMENT_ID',
'videoId' => 'VIDEO_ID',
'order' => [
'relevance',
'time',
],
],
[
'pageToken' => 'PAGE_TOKEN',
],
],
[
'playlists',
[
'part' => [
'snippet',
'statistics',
],
'id' => 'PLAYLIST_ID',
],
],
[
'playlistItems',
[
'part' => [
'snippet',
],
'playlistId' => 'PLAYLIST_ID',
],
[
'pageToken' => 'PAGE_TOKEN',
],
],
[
'search',
[
'part' => [
'id',
'snippet',
],
'q' => 'QUERY',
'channelId' => 'CHANNEL_ID',
'eventType' => 'upcoming',
'hashtag' => 'HASH_TAG',
'type' => [
'video',
'short',
],
'order' => [
'viewCount',
'relevance',
],
],
[
'pageToken' => 'PAGE_TOKEN',
],
],
[
'videos',
[
'part' => [
'id',
'status',
'contentDetails',
'music',
'short',
'impressions',
'musics',
'isPaidPromotion',
'isPremium',
'isMemberOnly',
'mostReplayed',
'qualities',
'captions',
'chapters',
'isOriginal',
'isRestricted',
'snippet',
'clip',
'activity',
'explicitLyrics',
'statistics',
],
'id' => 'VIDEO_ID',
'clipId' => 'CLIP_ID',
'SAPISIDHASH' => 'YOUR_SAPISIDHASH',
],
]
];
?>
<h1>YouTube operational API works when <?php yt(); ?> fails.</h1>
<h2>Current implemented features:</h2>
<?php
foreach ($features as $feature) {
feature($feature);
}
$features = [
[
'community',
[
'part' => [
'snippet',
],
'id' => 'POST_ID',
'channelId' => 'CHANNEL_ID',
'order' => [
'relevance',
'time',
],
],
],
[
'lives',
[
'part' => [
'donations',
'sponsorshipGifts',
'memberships',
'poll',
],
'id' => 'VIDEO_ID',
],
],
[
'liveChats',
[
'part' => [
'snippet',
'participants',
],
'id' => 'VIDEO_ID',
'time' => 'TIME_MS',
],
],
];
foreach ($features as $feature) {
echo "<p>" . url(WEBSITE_URL . "$feature[0]?" . getUrl($feature[1])) . "</p>";
}
?>
<h2>Make <?php yt(); ?> request WITHOUT ANY KEY:</h2>
<p>To make <strong>ANY <?php yt(); ?> request WITHOUT ANY KEY/USING YOUR QUOTA</strong>, you can use: <?php $noKey = 'noKey'; echo url(WEBSITE_URL . "$noKey/YOUR_REQUEST"); ?></p>
<p>For instance with <code>YOUR_REQUEST</code> being <code><?php $example = 'videos?part=snippet&id=VIDEO_ID'; echo $example; ?></code> you can use: <?php echo url(WEBSITE_URL . "$noKey/$example"); ?> instead of <?php echo url("https://www.googleapis.com/youtube/v3/$example"); ?></p>
<p>I may add in the future limitation per IP etc if the quota need to be better shared among the persons using this API.</p>
<?php
$keysCount = file_exists(KEYS_FILE) ? substr_count(file_get_contents(KEYS_FILE), "\n") + 1 : 0;
?>
<p>Currently this service is <a href='keys.php'>powered by <?php echo $keysCount; ?> keys</a>.</p>
<script>
function share() {
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
alert(xhttp.responseText);
youtubeDataAPIV3KeyInput.value = '';
}
};
var youtubeDataAPIV3KeyInput = document.getElementById('youtubeDataAPIV3Key');
const key = youtubeDataAPIV3KeyInput.value;
xhttp.open('GET', `addKey.php?key=${key}`);
xhttp.send();
}
</script>
<?php $YOUTUBE_DATA_API_V3_KEY_LENGTH = 39; ?>
Share your YouTube Data API v3 key to power the no-key service: <input type="text" id="youtubeDataAPIV3Key" placeholder="AIzaSy..." <?php printf('minlength="%s" maxlength="%s" size="%s"', $YOUTUBE_DATA_API_V3_KEY_LENGTH, $YOUTUBE_DATA_API_V3_KEY_LENGTH, $YOUTUBE_DATA_API_V3_KEY_LENGTH) ?>><button type="button" onClick="share()">share</button>
<h2>Open-source:</h2>
The source code is available on GitHub: <?php echo url('https://github.com/Benjamin-Loison/YouTube-operational-API'); ?>
<h2>Contact:</h2>
If a feature you are looking for which isn't working on <?php yt(); ?>, ask kindly with the below contact:<br/>
- <?php echo url('https://yt.lemnoslife.com/matrix', 'Matrix'); ?><br/>
- <?php echo url('https://yt.lemnoslife.com/discord', 'Discord'); ?>
<?php
$version = 'an unknown version.';
$ref = str_replace("\n", '', str_replace('ref: ', '', file_get_contents('.git/HEAD')));
$hash = file_get_contents(".git/$ref");
if ($hash !== false) {
$version = "version: <a href=\"https://github.com/Benjamin-Loison/YouTube-operational-API/commit/$hash\">$hash</a>";
}
echo "<br/><br/>This instance (" . SERVER_NAME . ") uses $version";
?>
</body>
</html>
================================================
FILE: keys.php
================================================
<!-- could add here stuff to add/remove key cf https://github.com/Benjamin-Loison/YouTube-operational-API/issues/18 -->
<!-- could also allow to share keys which have their daily quota spent -->
All keys have all worked at least once and some keys may have extended quota (more than 10,000).
================================================
FILE: liveChats.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
$liveTests = [];
include_once 'common.php';
$realOptions = [
'snippet',
'participants',
];
foreach ($realOptions as $realOption) {
$options[$realOption] = false;
}
if (isset($_GET['part'], $_GET['id']) && ($_GET['part'] != 'snippet' || isset($_GET['time']))) {
$part = $_GET['part'];
$parts = explode(',', $part, count($realOptions));
foreach ($parts as $part) {
if (!in_array($part, $realOptions)) {
dieWithJsonMessage("Invalid part $part");
} else {
$options[$part] = true;
}
}
if ($part == 'snippet' && !isPositiveInteger($_GET['time'])) {
dieWithJsonMessage('Invalid time');
}
$ids = $_GET['id'];
$realIds = explode(',', $ids);
verifyMultipleIds($realIds);
foreach ($realIds as $realId) {
if ((!isVideoId($realId))) {
dieWithJsonMessage('Invalid id');
}
}
echo getAPI($realIds);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function getItem($id)
{
global $options;
$result = getJSONFromHTML("https://www.youtube.com/watch?v=$id");
$continuation = $result['contents']['twoColumnWatchNextResults']['conversationBar']['liveChatRenderer']['continuations'][0]['reloadContinuationData']['continuation'];
$rawData = [
'context' => [
'client' => [
'clientName' => 'WEB',
'clientVersion' => MUSIC_VERSION
]
],
'continuation' => strval($continuation),
'currentPlayerState' => [
'playerOffsetMs' => $_GET['time']
]
];
$opts = [
'http' => [
'header' => ['Content-Type: application/json'],
'method' => 'POST',
'content' => json_encode($rawData),
]
];
$result = getJSON('https://www.youtube.com/youtubei/v1/live_chat/get_live_chat_replay?key=' . UI_KEY, $opts);
$item = [
'kind' => 'youtube#video',
'etag' => 'NotImplemented',
'id' => $id
];
if ($options['snippet']) {
$snippet = [];
$actions = $result['continuationContents']['liveChatContinuation']['actions'];
foreach ($actions as $action) {
$replayChatItemAction = $action['replayChatItemAction'];
$liveChatTextMessageRenderer = $replayChatItemAction['actions'][0]['addChatItemAction']['item']['liveChatTextMessageRenderer'];
if ($liveChatTextMessageRenderer != null) {
$message = [
'id' => urldecode($liveChatTextMessageRenderer['id']),
'message' => $liveChatTextMessageRenderer['message']['runs'],
'authorName' => $liveChatTextMessageRenderer['authorName']['simpleText'],
'authorThumbnails' => $liveChatTextMessageRenderer['authorPhoto']['thumbnails'],
'timestampAbsoluteUsec' => intval($liveChatTextMessageRenderer['timestampUsec']),
'authorChannelId' => $liveChatTextMessageRenderer['authorExternalChannelId'],
'timestamp' => getIntFromDuration($liveChatTextMessageRenderer['timestampText']['simpleText']),
'videoOffsetTimeMsec' => intval($replayChatItemAction['videoOffsetTimeMsec'])
];
array_push($snippet, $message);
}
}
$item['snippet'] = $snippet;
}
if ($options['participants']) {
$participants = [];
$opts = [
'http' => [
'header' => ['User-Agent: ' . USER_AGENT],
]
];
$result = getJSONFromHTML("https://www.youtube.com/live_chat?continuation=$continuation", $opts, 'window["ytInitialData"]', '');
$participants = array_slice($result['continuationContents']['liveChatContinuation']['actions'], 1);
$item['participants'] = $participants;
}
return $item;
}
function getAPI($ids)
{
$items = [];
foreach ($ids as $id) {
array_push($items, getItem($id));
}
$answer = [
'kind' => 'youtube#videoListResponse',
'etag' => 'NotImplemented',
'items' => $items
];
return json_encode($answer, JSON_PRETTY_PRINT);
}
================================================
FILE: lives.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
$liveTests = [];
include_once 'common.php';
$realOptions = [
'donations',
'sponsorshipGifts',
'memberships',
'poll',
];
foreach ($realOptions as $realOption) {
$options[$realOption] = false;
}
if (isset($_GET['part'], $_GET['id'])) {
$part = $_GET['part'];
$parts = explode(',', $part, count($realOptions));
foreach ($parts as $part) {
if (!in_array($part, $realOptions)) {
dieWithJsonMessage("Invalid part $part");
} else {
$options[$part] = true;
}
}
$ids = $_GET['id'];
$realIds = explode(',', $ids);
verifyMultipleIds($realIds);
foreach ($realIds as $realId) {
if ((!isVideoId($realId))) {
dieWithJsonMessage('Invalid id');
}
}
echo getAPI($realIds);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function getItem($id)
{
global $options;
$opts = [
'http' => [
'user_agent' => USER_AGENT,
'header' => ['Accept-Language: en'],
]
];
$result = getJSONFromHTML("https://www.youtube.com/live_chat?v=$id", $opts, 'window["ytInitialData"]', '');
$item = [
'kind' => 'youtube#video',
'etag' => 'NotImplemented',
'id' => $id
];
$actions = $result['contents']['liveChatRenderer']['actions'];
if ($options['donations']) {
$donations = [];
foreach ($actions as $action) {
$donation = $action['addLiveChatTickerItemAction']['item']['liveChatTickerPaidMessageItemRenderer']['showItemEndpoint']['showLiveChatItemEndpoint']['renderer']['liveChatPaidMessageRenderer'];
if ($donation != null) {
array_push($donations, $donation);
}
}
$item['donations'] = $donations;
}
if ($options['sponsorshipGifts']) {
function getCleanAuthorBadge($authorBadgeRaw)
{
$liveChatAuthorBadgeRenderer = $authorBadgeRaw['liveChatAuthorBadgeRenderer'];
$authorBadge = [
'tooltip' => $liveChatAuthorBadgeRenderer['tooltip'],
'customThumbnail' => $liveChatAuthorBadgeRenderer['customThumbnail']['thumbnails']
];
return $authorBadge;
}
function cleanMembershipOrSponsorship($raw, $isMembership) {
$common = $isMembership ? $raw : $raw['header']['liveChatSponsorshipsHeaderRenderer'];
$primaryText = implode('', array_map(fn($run) => $run['text'], $common[$isMembership ? 'headerPrimaryText' : 'primaryText']['runs']));
$subText = $raw['headerSubtext']['simpleText'];
$authorBadges = array_map('getCleanAuthorBadge', $common['authorBadges']);
$clean = [
'id' => $raw['id'],
'timestamp' => intval($raw['timestampUsec']),
'authorChannelId' => $raw['authorExternalChannelId'],
'authorName' => $common['authorName']['simpleText'],
'authorPhoto' => $common['authorPhoto']['thumbnails'],
'primaryText' => $primaryText,
'subText' => $subText,
'authorBadges' => $authorBadges,
];
return $clean;
}
$sponsorshipGifts = [];
foreach ($actions as $action) {
$sponsorshipGift = $action['addChatItemAction']['item']['liveChatSponsorshipsGiftPurchaseAnnouncementRenderer'];
if ($sponsorshipGift != null)
{
array_push($sponsorshipGifts, cleanMembershipOrSponsorship($sponsorshipGift, false));
}
}
$item['sponsorshipGifts'] = $sponsorshipGifts;
}
if ($options['memberships']) {
$memberships = [];
foreach ($actions as $action) {
$membership = $action['addChatItemAction']['item']['liveChatMembershipItemRenderer'];
if ($membership != null)
{
array_push($memberships, cleanMembershipOrSponsorship($membership, true));
}
}
$item['memberships'] = $memberships;
}
if ($options['poll']) {
$firstAction = $actions[0];
if(array_key_exists('showLiveChatActionPanelAction', $firstAction)) {
$pollRenderer = $firstAction['showLiveChatActionPanelAction']['panelToShow']['liveChatActionPanelRenderer']['contents']['pollRenderer'];
$pollHeaderRenderer = $pollRenderer['header']['pollHeaderRenderer'];
$liveChatPollStateEntity = $result['frameworkUpdates']['entityBatchUpdate']['mutations'][0]['payload']['liveChatPollStateEntity'];
$metadataTextRuns = explode(' • ', $liveChatPollStateEntity['metadataText']['runs'][0]['text']);
$poll = [
'question' => $liveChatPollStateEntity['collapsedMetadataText']['runs'][2]['text'],
'choices' => array_map(fn($choiceText, $choiceRatio) => [
'text' => $choiceText['text']['runs'][0]['text'],
'voteRatio' => $choiceRatio['value']['voteRatio'],
], $pollRenderer['choices'], $liveChatPollStateEntity['pollChoiceStates']),
'channelName' => $metadataTextRuns[0],
'timestamp' => str_replace("\u{00a0}", ' ', $metadataTextRuns[1]),
'totalVotes' => intval(str_replace(' votes', '', $metadataTextRuns[2])),
'channelThumbnails' => $pollHeaderRenderer['thumbnail']['thumbnails'],
];
}
$item['poll'] = $poll;
}
return $item;
}
function getAPI($ids)
{
$items = [];
foreach ($ids as $id) {
array_push($items, getItem($id));
}
$answer = [
'kind' => 'youtube#videoListResponse',
'etag' => 'NotImplemented',
'items' => $items
];
return json_encode($answer, JSON_PRETTY_PRINT);
}
================================================
FILE: noKey/.htaccess
================================================
Options +FollowSymLinks
RewriteEngine on
RewriteRule ^.*$ index.php
================================================
FILE: noKey/index.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
chdir('..');
include_once 'common.php';
$requestUri = $_SERVER['REQUEST_URI'];
// As YouTube Data API v3 considers only the first passed `key` parameter if there are multiple of them, providing a first incorrect key convince the no-key service that all its keys are incorrect.
if(str_contains($requestUri, 'key='))
dieWithJsonMessage('No YouTube Data API v3 key is required to use the no-key service!');
if(!file_exists(KEYS_FILE))
dieWithJsonMessage(KEYS_FILE . ' does not exist!');
$content = file_get_contents(KEYS_FILE);
$keys = explode("\n", $content);
$keysCount = count($keys);
$parts = explode('/noKey/', $requestUri);
$url = 'https://www.googleapis.com/youtube/v3/' . end($parts) . '&key=';
$options = ['http' => ['ignore_errors' => true]];
$context = stream_context_create($options);
function myDie($content)
{
global $keysCount;
if(isset($_GET['monitoring']))
{
$data = json_decode($content, true);
$data['monitoring'] = $keysCount;
$content = json_encode($data, JSON_PRETTY_PRINT);
}
die($content);
}
/// is there any way someone may get the keys out ? could restrict syntax with the one of the official API but that's not that much clean
// Tries to proceed to the request with an API key and if running out of quota, then use for this and following requests the API key used the longest time ago.
for ($keysIndex = 0; $keysIndex < $keysCount; $keysIndex++) {
$key = $keys[$keysIndex];
$realUrl = $url . $key;
$response = file_get_contents($realUrl, false, $context);
$response = str_replace($key, '!Please contact Benjamin Loison to tell him how you did that!', $response); // quite good but not perfect
// no need to check for ip leak
$json = json_decode($response, true);
if (array_key_exists('error', $json)) {
$error = $json['error'];
if ($error['errors'][0]['domain'] !== 'youtube.quota') {
$message = $error['message'];
// As there are many different kind of errors other than the quota one, we could just proceed to a test verifying that the expected result is returned, as when adding a key.
if ($message === 'API key expired. Please renew the API key.' or str_ends_with($message, 'has been suspended.') or $message === 'API key not valid. Please pass a valid API key.' or $message === 'API Key not found. Please pass a valid API key.' or str_starts_with($message, 'YouTube Data API v3 has not been used in project ') or str_ends_with($message, 'are blocked.') or checkRegex('The provided API key has an IP address restriction\. The originating IP address of the call \(([\da-f:]{4,39}|\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\) violates this restriction\.', $message)) {
// Removes this API key as it won't be useful anymore.
$newKeys = array_merge(array_slice($keys, $keysIndex + 1), array_slice($keys, 0, $keysIndex));
$toWrite = implode("\n", $newKeys);
file_put_contents(KEYS_FILE, $toWrite);
// Skips to next API key.
// Decrements `keysIndex` as it will be incremented due to `continue`.
$keysIndex -= 1;
$keysCount -= 1;
$keys = $newKeys;
continue;
}
// If such an error occur, returns it to the end-user, as made exceptions for out of quota and expired keys, should also consider transient, backend and suspension errors.
// As managed in YouTube-comments-graph: https://github.com/Benjamin-Loison/YouTube-comments-graph/blob/993429770417bdfa4fdf176c473ff1bfe7ed21ae/CPP/main.cpp#L55-L60
myDie($response);
}
} else {
if ($keysIndex !== 0) {
// As the request is successful with this API key, prioritize this key and all the following ones over the first ones.
$newKeys = array_merge(array_slice($keys, $keysIndex), array_slice($keys, 0, $keysIndex));
$toWrite = implode("\n", $newKeys);
file_put_contents(KEYS_FILE, $toWrite);
}
// Returns the proceeded response to the end-user.
myDie($response);
}
}
$message = 'The request cannot be completed because the YouTube operational API run out of quota. Please try again later.';
$errors = [
'message' => $message,
'domain' => 'youtube.quota',
'reason' => 'quotaExceeded'
];
$error = [
'code' => 403,
'message' => $message,
'errors' => [$errors]
];
$json = ['error' => $error];
myDie(json_encode($json, JSON_PRETTY_PRINT));
================================================
FILE: playlistItems.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
$playlistItemsTests = [
// not precise...
// Preferably have only more than 200 different videos but that I own would be more robust
//['part=snippet&playlistId=PLKAl8tt2R8OfMnDRnEABZ2M-tI7yJYvl1', 'items/0/snippet/publishedAt', 1520963713]
];
include_once 'common.php';
if (isset($_GET['part'], $_GET['playlistId'])) {
$part = $_GET['part'];
if (!in_array($part, ['snippet'])) {
dieWithJsonMessage('Invalid part');
}
$playlistId = $_GET['playlistId'];
if (!isPlaylistId($playlistId)) {
dieWithJsonMessage('Invalid playlistId');
}
$continuationToken = '';
if (isset($_GET['pageToken'])) {
$continuationToken = $_GET['pageToken'];
if (!isContinuationToken($continuationToken)) {
dieWithJsonMessage('Invalid pageToken');
}
}
echo getAPI($playlistId, $continuationToken);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function getAPI($playlistId, $continuationToken)
{
$continuationTokenProvided = $continuationToken != '';
$http = [];
$url = '';
if ($continuationTokenProvided) {
$url = 'https://www.youtube.com/youtubei/v1/browse?key=' . UI_KEY;
$rawData = [
'context' => [
'client' => [
'clientName' => 'WEB',
'clientVersion' => MUSIC_VERSION
]
],
'continuation' => $continuationToken
];
$http['method'] = 'POST';
$http['header'] = ['Content-Type: application/json'];
$http['content'] = json_encode($rawData);
} else {
$url = "https://www.youtube.com/playlist?list=$playlistId";
$http['header'] = ['Accept-Language: en'];
}
$httpOptions = [
'http' => $http
];
if ($continuationTokenProvided) {
$result = getJSON($url, $httpOptions);
} else {
$result = getJSONFromHTML($url, $httpOptions);
}
$answerItems = [];
$items = $continuationTokenProvided ? getContinuationItems($result) : getTabs($result)[0]['tabRenderer']['content']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'][0]['playlistVideoListRenderer']['contents'];
$itemsCount = count($items);
for ($itemsIndex = 0; $itemsIndex < $itemsCount - 1; $itemsIndex++) {
$item = $items[$itemsIndex];
$playlistVideoRenderer = $item['playlistVideoRenderer'];
$videoId = $playlistVideoRenderer['videoId'];
$title = $playlistVideoRenderer['title']['runs'][0]['text'];
$publishedAt = getPublishedAt($playlistVideoRenderer['videoInfo']['runs'][2]['text']);
$thumbnails = $playlistVideoRenderer['thumbnail']['thumbnails'];
$answerItem = [
'kind' => 'youtube#playlistItem',
'etag' => 'NotImplemented',
'snippet' => [
'publishedAt' => $publishedAt,
'title' => $title,
'thumbnails' => $thumbnails,
'resourceId' => [
'kind' => 'youtube#video',
'videoId' => $videoId
]
]
];
array_push($answerItems, $answerItem);
}
$nextContinuationToken = urldecode($items[100]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token']); // it doesn't seem random but hard to reverse-engineer
$answer = [
'kind' => 'youtube#playlistItemListResponse',
'etag' => 'NotImplemented'
];
// order matter or could afterwards sort by an (official YT API) arbitrary order (not alphabetical)
// seems to be this behavior with the official API
if ($nextContinuationToken != '') {
$answer['nextPageToken'] = $nextContinuationToken;
}
$answer['items'] = $answerItems;
return json_encode($answer, JSON_PRETTY_PRINT);
}
================================================
FILE: playlists.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
$playlistsTests = [
['part=snippet&id=PL8wZFyWE1ZaI2HE7PYHvpx0_yv4oJjwAZ', 'items/0/snippet/title', '4,000 times the same video'],
['part=statistics&id=PL8wZFyWE1ZaI2HE7PYHvpx0_yv4oJjwAZ', 'items/0/statistics', ['videoCount' => 4_000]],
];
include_once 'common.php';
$realOptions = [
'snippet',
'statistics',
];
// really necessary ?
foreach ($realOptions as $realOption) {
$options[$realOption] = false;
}
if (isset($_GET['part'], $_GET['id'])) {
$part = $_GET['part'];
$parts = explode(',', $part, count($realOptions));
foreach ($parts as $part) {
if (!in_array($part, $realOptions)) {
dieWithJsonMessage("Invalid part $part");
} else {
$options[$part] = true;
}
}
$ids = $_GET['id'];
$realIds = explode(',', $ids);
verifyMultipleIds($realIds);
foreach ($realIds as $realId) {
if (!isPlaylistId($realId)) {
dieWithJsonMessage('Invalid id');
}
}
echo getAPI($realIds);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function getItem($id)
{
global $options;
$result = getJSONFromHTML("https://www.youtube.com/playlist?list=$id", forceLanguage: true);
$item = [
'kind' => 'youtube#playlist',
'etag' => 'NotImplemented'
];
if ($options['snippet']) {
$title = $result['metadata']['playlistMetadataRenderer']['title'];
$item['snippet'] = [
'title' => $title
];
}
if ($options['statistics']) {
$viewCount = $result['sidebar']['playlistSidebarRenderer']['items'][0]['playlistSidebarPrimaryInfoRenderer']['stats'][1]['simpleText'];
$viewCount = getIntFromViewCount($viewCount);
$videoCount = intval(str_replace(',', '', $result['header']['playlistHeaderRenderer']['numVideosText']['runs'][0]['text']));
$item['statistics'] = [
'viewCount' => $viewCount,
'videoCount' => $videoCount
];
}
return $item;
}
function getAPI($ids)
{
$items = [];
foreach ($ids as $id) {
array_push($items, getItem($id));
}
$answer = [
'kind' => 'youtube#playlistListResponse',
'etag' => 'NotImplemented',
'items' => $items
];
return json_encode($answer, JSON_PRETTY_PRINT);
}
================================================
FILE: proto/php/.gitignore
================================================
# Ignore everything in this directory
*
# Except this file
!.gitignore
================================================
FILE: proto/prototypes/browse.proto
================================================
syntax = "proto3";
message Browse {
string endpoint = 2;
SubBrowse subBrowse = 25;
}
message SubBrowse {
string postId = 22;
}
================================================
FILE: proto/prototypes/browse_shorts.proto
================================================
syntax = "proto3";
message BrowseShorts {
Sub0BrowseShorts two = 2;
int32 three = 3;
string four = 4;
}
message Sub0BrowseShorts {
string two = 2;
string three = 3;
}
message Sub1BrowseShorts {
Sub2BrowseShorts two = 2;
}
message Sub2BrowseShorts {
Sub3BrowseShorts eighteen = 18;
}
message Sub3BrowseShorts {
Sub4_7BrowseShorts seven = 7;
Sub4_9BrowseShorts nine = 9;
}
message Sub4_7BrowseShorts {
int32 twelve = 12;
}
message Sub4_9BrowseShorts {
}
================================================
FILE: search.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
// Stack Overflow source: https://stackoverflow.com/a/70793047
$searchTests = [
//['part=snippet&channelId=UC4QobU6STFB0P71PMvOGN5A&order=viewCount', 'items/0/id/videoId', 'jNQXAC9IVRw'],
];
// copy YT perfectly (answers and arguments) - slower because not always everything from answer in one request for me
// make an API based on one request I receive involves one request on my side - more precise in terms of complexity
// can from this last model also just include "the interesting data" and nothing repetitive with the YouTube Data API v3, I mean that from the videoId we can get all details we want from the official API so maybe no need to repeat some here even if there are in the answer of my request
include_once 'common.php';
includeOnceProtos([
'BrowseShorts',
'Sub0BrowseShorts',
'Sub1BrowseShorts',
'Sub2BrowseShorts',
'Sub3BrowseShorts',
'Sub4_7BrowseShorts',
'Sub4_9BrowseShorts',
]);
$realOptions = [
'id',
'snippet',
];
// really necessary ?
foreach ($realOptions as $realOption) {
$options[$realOption] = false;
}
if (isset($_GET['part']) &&
(isset($_GET['channelId']) || isset($_GET['channelId'], $_GET['eventType']) || isset($_GET['hashtag']) || isset($_GET['q'])) &&
(isset($_GET['order']) || isset($_GET['hashtag']) || isset($_GET['q']) || isset($_GET['eventType']))) {
$part = $_GET['part'];
$parts = explode(',', $part, count($realOptions));
foreach ($parts as $part) {
if (!in_array($part, $realOptions)) {
dieWithJsonMessage("Invalid part $part");
} else {
$options[$part] = true;
}
}
if ($options['snippet']) {
$options['id'] = true;
}
$id = '';
if (isset($_GET['channelId'])) {
$id = $_GET['channelId'];
if (!isChannelId($id)) {
dieWithJsonMessage('Invalid channelId');
}
} elseif ($_GET['eventType']) {
$eventType = $_GET['eventType'];
if (!isEventType($eventType)) {
dieWithJsonMessage('Invalid eventType');
}
} elseif ($_GET['hashtag']) {
$id = $_GET['hashtag'];
if (!isHashtag($id)) {
dieWithJsonMessage('Invalid hashtag');
}
} elseif ($_GET['q']) {
$id = $_GET['q'];
if (!isQuery($id)) {
dieWithJsonMessage('Invalid q');
}
} else {
dieWithJsonMessage('No channelId or hashtag or q field was provided');
}
if ((isset($_GET['order'])) && !isset($_GET['eventType'])) {
$order = $_GET['order'];
if (!in_array($order, ['viewCount', 'relevance'])) {
dieWithJsonMessage('Invalid order');
}
}
$continuationToken = '';
if (isset($_GET['pageToken'])) {
$continuationToken = $_GET['pageToken'];
// what checks to do ?
if (!isContinuationToken($continuationToken)) {
dieWithJsonMessage('Invalid pageToken');
}
}
echo getAPI($id, $order, $continuationToken);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function getAPI($id, $order, $continuationToken)
{
global $options;
$items = null;
$continuationTokenProvided = $continuationToken != '';
if (isset($_GET['hashtag'])) {
if ($continuationTokenProvided) {
$json = getContinuationJson($continuationToken);
} else {
$json = getJSONFromHTML('https://www.youtube.com/hashtag/' . urlencode($id));
}
$items = $continuationTokenProvided ? getContinuationItems($json) : getTabs($json)[0]['tabRenderer']['content']['richGridRenderer']['contents'];
} elseif (isset($_GET['eventType'])) {
$json = getJSONFromHTML("https://www.youtube.com/channel/{$_GET['channelId']}/videos?view=2&live_view=502");
$items = getTabs($json)[1]['tabRenderer']['content']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'][0]['gridRenderer']['items'];
} elseif (isset($_GET['q'])) {
$typeBase64 = $order === 'relevance' ? '' : 'EgIQAQ==';
$rawData = [
'context' => [
'client' => [
'clientName' => 'WEB',
'clientVersion' => MUSIC_VERSION
]
],
];
if(isset($_GET['type']) && $_GET['type'] === 'short') {
$sub1BrowseShorts = (new \Sub1BrowseShorts())
->setTwo((new \Sub2BrowseShorts())
->setEighteen((new \Sub3BrowseShorts())
->setSeven((new \Sub4_7BrowseShorts())
->setTwelve(26))
->setNine(new \Sub4_9BrowseShorts())));
$browseShorts = (new \BrowseShorts())
->setTwo((new \Sub0BrowseShorts())
->setTwo($_GET['q'])
->setThree(base64_encode($sub1BrowseShorts->serializeToString())))
->setThree(52047873)
->setFour('search-page');
$continuation = base64url_encode($browseShorts->serializeToString());
$rawData['continuation'] = $continuation;
} else {
$rawData['query'] = str_replace('"', '\"', $_GET['q']);
if($typeBase64 !== '') {
$rawData['params'] = $typeBase64;
}
}
if($continuationTokenProvided) {
$rawData['continuation'] = $continuationToken;
}
$opts = [
'http' => [
'method' => 'POST',
'header' => ['Content-Type: application/json'],
'content' => json_encode($rawData),
]
];
$json = getJSON('https://www.youtube.com/youtubei/v1/search?key=' . UI_KEY, $opts);
if(isset($_GET['type']) && $_GET['type'] === 'short')
{
$items = $json['onResponseReceivedCommands'][0]['reloadContinuationItemsCommand']['continuationItems'][0]['twoColumnSearchResultsRenderer']['primaryContents']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'];
}
else
{
$items = ($continuationTokenProvided ? $json['onResponseReceivedCommands'][0]['appendContinuationItemsAction']['continuationItems'] : $json['contents']['twoColumnSearchResultsRenderer']['primaryContents']['sectionListRenderer']['contents'])[0]['itemSectionRenderer']['contents'];
}
} else { // if (isset($_GET['channelId']))
$orderBase64 = 'EgZ2aWRlb3MYASAAMAE=';
$rawData = [
'context' => [
'client' => [
'clientName' => 'WEB',
'clientVersion' => CLIENT_VERSION
]
],
'browseId' => $_GET['channelId'],
'params' => $orderBase64
];
if($continuationTokenProvided) {
$rawData['continuation'] = $continuationToken;
}
$opts = [
'http' => [
'method' => 'POST',
'header' => ['Content-Type: application/json'],
'content' => json_encode($rawData),
]
];
$result = getJSON('https://www.youtube.com/youtubei/v1/browse?key=' . UI_KEY, $opts);
// repeated on official API but not in UI requests
//if(!$continuationTokenProvided)
// $regionCode = $result['topbar']['desktopTopbarRenderer']['countryCode'];
$items = $continuationTokenProvided ? getContinuationItems($result) : getTabs($result)[1]['tabRenderer']['content']['sectionListRenderer']['contents'][0]['itemSectionRenderer']['contents'][0]['gridRenderer']['items'];
}
$answerItems = [];
$itemsCount = count($items);
for ($itemsIndex = 0; $itemsIndex < $itemsCount - ($continuationTokenProvided || $_GET['hashtag'] ? 1 : 0); $itemsIndex++) { // check upper bound for hashtags
$item = $items[$itemsIndex];
$path = '';
if (isset($_GET['hashtag'])) {
$path = 'richItemRenderer/content/videoRenderer';
} elseif (isset($_GET['q'])) {
$path = 'videoRenderer';
// Skip `People also watched`.
if(!array_key_exists($path, $item)) {
continue;
}
} else {
$path = 'gridVideoRenderer';
}
$gridVideoRenderer = getValue($item, $path);
$answerItem = [
'kind' => 'youtube#searchResult',
'etag' => 'NotImplemented'
];
if ($options['id']) {
$videoId = $gridVideoRenderer['videoId'];
$answerItem['id'] = [
'kind' => 'youtube#video',
'videoId' => $videoId
];
}
if ($options['snippet']) {
$title = $gridVideoRenderer['title']['runs'][0]['text'];
$run = $gridVideoRenderer['ownerText']['runs'][0];
$browseEndpoint = $run['navigationEndpoint']['browseEndpoint'];
$channelId = $browseEndpoint['browseId'];
$views = getIntFromViewCount($gridVideoRenderer['viewCountText']['simpleText']);
$badges = $gridVideoRenderer['badges'];
$badges = !empty($badges) ? array_map(fn($badge) => $badge['metadataBadgeRenderer']['label'], $badges) : [];
$chapters = $gridVideoRenderer['expandableMetadata']['expandableMetadataRenderer']['expandedContent']['horizontalCardListRenderer']['cards'];
$chapters = !empty($chapters) ? array_map(function($chapter) {
$macroMarkersListItemRenderer = $chapter['macroMarkersListItemRenderer'];
return [
'title' => $macroMarkersListItemRenderer['title']['simpleText'],
'time' => getIntFromDuration($macroMarkersListItemRenderer['timeDescription']['simpleText']),
'thumbnails' => $macroMarkersListItemRenderer['thumbnail']['thumbnails']
]; }, $chapters) : [];
$channelHandle = substr($browseEndpoint['canonicalBaseUrl'], 1);
$answerItem['snippet'] = [
'channelId' => $channelId,
'title' => $title,
'thumbnails' => $gridVideoRenderer['thumbnail']['thumbnails'],
'channelTitle' => $run['text'],
'channelHandle' => $channelHandle[0] === '@' ? $channelHandle : null,
'timestamp' => $gridVideoRenderer['publishedTimeText']['simpleText'],
'duration' => getIntFromDuration($gridVideoRenderer['lengthText']['simpleText']),
'views' => $views,
'badges' => $badges,
'channelApproval' => $gridVideoRenderer['ownerBadges'][0]['metadataBadgeRenderer']['tooltip'],
'channelThumbnails' => $gridVideoRenderer['channelThumbnailSupportedRenderers']['channelThumbnailWithLinkRenderer']['thumbnail']['thumbnails'],
'detailedMetadataSnippet' => $gridVideoRenderer['detailedMetadataSnippets'][0]['snippetText']['runs'],
'chapters' => $chapters
];
}
array_push($answerItems, $answerItem);
}
if (isset($_GET['hashtag'])) {
$nextContinuationToken = $itemsCount > 60 ? $items[60] : '';
} else {
$nextContinuationToken = $itemsCount > 30 ? $items[30] : '';
} // it doesn't seem random but hard to reverse-engineer
if ($nextContinuationToken !== '') {
$nextContinuationToken = $nextContinuationToken['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'];
}
if (isset($_GET['q'])) {
$nextContinuationToken = ($continuationTokenProvided ? $json['onResponseReceivedCommands'][0]['appendContinuationItemsAction']['continuationItems'] : $json['contents']['twoColumnSearchResultsRenderer']['primaryContents']['sectionListRenderer']['contents'])[1]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'];
}
$nextContinuationToken = urldecode($nextContinuationToken);
$answer = [
'kind' => 'youtube#searchListResponse',
'etag' => 'NotImplemented'
];
// order matter or could afterwards sort by an (official YT API) arbitrary order (not alphabetical)
// seems to be this behavior with the official API
if ($nextContinuationToken != '') {
$answer['nextPageToken'] = $nextContinuationToken;
}
//if(!$continuationTokenProvided) // doesn't seem accurate
// $answer['regionCode'] = $regionCode;
$answer['items'] = $answerItems;
return json_encode($answer, JSON_PRETTY_PRINT);
}
================================================
FILE: tools/checkOperationnalAPI.py
================================================
#!/usr/bin/env python
import requests
query = 'test'
videoIds = []
def getVideos(pageToken = '', callsIndex = 0):
global videoIds
url = f'http://localhost/search?part=id&q={query}&type=video'
if pageToken != '':
url += '&pageToken=' + pageToken
data = requests.get(url).json()
for item in data['items']:
videoId = item['id']['videoId']
if not videoId in videoIds:
videoIds += [videoId]
print(len(videoIds), callsIndex)
if 'nextPageToken' in data:
getVideos(data['nextPageToken'], callsIndex + 1)
getVideos()
================================================
FILE: tools/getJSONPathFromKey.py
================================================
#!/usr/bin/env python
'''
This script purpose is to ease retrieving the JSON path associated to an interested YouTube data entry.
For instance when looking for a feature on YouTube UI let say a YouTube video title that we want to automate the retrieval we plug as `filePath` of this script the returned YouTube UI HTML. This script will extract and update to the provided `filePath` the relevant JSON encoded in the appropriate JavaScript variable. Then this script looks recursively for the entry concerning the specific video title you are looking for.
For instance:
```bash
curl -s 'https://www.youtube.com/watch?v=jNQXAC9IVRw' > jNQXAC9IVRw.html
./getJSONPathFromKey.py jNQXAC9IVRw.html | grep 'Me at the zoo$'
```
```
105 /contents/twoColumnWatchNextResults/results/results/contents/0/videoPrimaryInfoRenderer/title/runs/0/text Me at the zoo
101 /playerOverlays/playerOverlayRenderer/videoDetails/playerOverlayVideoDetailsRenderer/title/simpleText Me at the zoo
156 /engagementPanels/2/engagementPanelSectionListRenderer/content/structuredDescriptionContentRenderer/items/0/videoDescriptionHeaderRenderer/title/runs/0/text Me at the zoo
170 /engagementPanels/2/engagementPanelSectionListRenderer/content/structuredDescriptionContentRenderer/items/3/reelShelfRenderer/items/0/reelItemRenderer/headline/simpleText Me at the zoo
171 /engagementPanels/2/engagementPanelSectionListRenderer/content/structuredDescriptionContentRenderer/items/3/reelShelfRenderer/items/24/reelItemRenderer/headline/simpleText этому видио 17 лет - Me at the zoo
```
The first number is the path length to ease considering the shortest one.
If for some reason you know the entry name but not the path you can provide the optional `entryName` `./getJSONPathFromKey.py jNQXAC9IVRw.html entryName` to get the whole path.
As there are potentially multiple JavaScript variable names you can provide as the third argument the interesting JavaScript variable name.
'''
import sys
import json
from lxml import html
def treatKey(obj, path, key):
objKey = obj[key]
objKeyType = type(objKey)
value = objKey if (not objKeyType is dict and not objKeyType is list) else ''
# used to be a print
return (path + '/' + key, value)
def _finditem(obj, key, path = ''):
objType = type(obj)
results = []
if objType is dict:
keys = obj.keys()
if key == '':
for keyTmp in keys:
results += [treatKey(obj, path, keyTmp)]
elif key in keys:
results += [treatKey(obj, path, key)]
for keyTmp in keys:
res = _finditem(obj[keyTmp], key, path + '/' + keyTmp)
if res != []:
results += res
elif objType is list:
objLen = len(obj)
for objIndex in range(objLen):
objEl = obj[objIndex]
res = _finditem(objEl, key, path + '/' + str(objIndex))
if res != []:
results += res
return results
filePath = sys.argv[1]
key = sys.argv[2] if len(sys.argv) >= 3 else ''
# `ytVariableName` could be for instance 'ytInitialPlayerResponse'
ytVariableName = sys.argv[3] if len(sys.argv) >= 4 else 'ytInitialData'
# if not found from key could search by value
# that way could find easily shortest path to get the value as sometimes the value is repeated multiple times
with open(filePath) as f:
try:
json.load(f)
isJSON = True
except:
isJSON = False
if not isJSON:
with open(filePath) as f:
content = f.read()
# Should use a JavaScript parser instead of proceeding that way.
# Same comment concerning `getJSONStringFromHTMLScriptPrefix`, note that both parsing methods should be identical.
tree = html.fromstring(content)
ytVariableDeclaration = ytVariableName + ' = '
for script in tree.xpath('//script'):
scriptContent = script.text_content()
if ytVariableDeclaration in scriptContent:
newContent = scriptContent.split(ytVariableDeclaration)[1][:-1]
break
with open(filePath, 'w') as f:
f.write(newContent)
with open(filePath) as f:
data = json.load(f)
with open(filePath, 'w', encoding='utf-8') as f:
json.dump(data, f, ensure_ascii = False, indent = 4)
pathValues = _finditem(data, key)
if pathValues != []:
longestPath = len(str(max([len(path) for path, _ in pathValues])))
for path, value in pathValues:
pathLength = ' ' * (longestPath - len(str(len(path)))) + str(len(path))
print(pathLength, path, value)
================================================
FILE: tools/minimizeCURL.py
================================================
#!/usr/bin/env python
## /!\ Assume that the content of `curlCommandFilePath` is trusted /!\
# TODO: precising or/and lowering this trust level would be interesting
# Note that this algorithm doesn't currently minimize specifically YouTube requests (like `--data-raw/context/client/clientVersion`), should potentially add in the future useless character removal which would be a more general but incomplete approach.
'''
For the moment this algorithm only removes unnecessary:
- headers
- URL parameters
- cookies
- raw data
'''
import shlex
import subprocess
import json
import copy
import sys
from urllib.parse import urlparse, parse_qs, quote_plus
import re
import os
# Could precise the input file and possibly remove the output one as the minimized requests start to be short.
if len(sys.argv) < 3:
print('Usage: ./minimizeCURL curlCommand.txt "Wanted output"')
exit(1)
curlCommandFilePath = sys.argv[1]
wantedOutput = sys.argv[2].encode('utf-8')
# The purpose of these parameters is to reduce requests done when developing this script:
removeHeaders = True
removeUrlParameters = True
removeCookies = True
removeRawData = True
PRINT_TRY_TO_REMOVE = False
VERIFY_INITIAL_COMMAND = True
def printTryToRemove(toRemove):
if PRINT_TRY_TO_REMOVE:
print(f'Try to remove: {toRemove}!')
# Pay attention to provide a command giving plaintext output, so might required to remove `Accept-Encoding` HTTPS header.
with open(curlCommandFilePath) as f:
command = f.read()
def executeCommand(command):
# `stderr = subprocess.DEVNULL` is used to get rid of curl progress.
# Could also add `-s` curl argument.
try:
result = subprocess.check_output(command, shell = True, stderr = subprocess.DEVNULL)
except:
return b''
return result
def getCommandScript(fileName):
return f'{fileName}.sh'
def writeCommand(fileName, command):
with open(getCommandScript(fileName), 'w') as f:
f.write(command)
PARTIALLY_MINIMIZED_CURL_SCRIPT_NAME = 'partially_minimized_curl'
def isCommandStillFine(command):
result = executeCommand(command)
isCommandStillFineResult = wantedOutput in result
# [Benjamin-Loison/cpython/issues/48](https://github.com/Benjamin-Loison/cpython/issues/48)
if isCommandStillFineResult:
writeCommand(PARTIALLY_MINIMIZED_CURL_SCRIPT_NAME, command)
return isCommandStillFineResult
def getCommandLengthFormatted(command):
return f'{len(command):,}'
def printThatCommandIsStillFine(command):
print(f'Command with length {getCommandLengthFormatted(command)} is still fine.')
# For Chromium support:
command = command.replace(' \\\n ', '')
print(f'Initial command length: {getCommandLengthFormatted(command)}.')
# To verify that the user provided the correct `wantedOutput` to keep during the minimization.
if VERIFY_INITIAL_COMMAND and not isCommandStillFine(command):
print('The wanted output isn\'t contained in the result of the original curl command!')
exit(1)
if removeHeaders:
print('Removing headers')
# Should try to minimize the number of requests done, by testing half of parameters at each request.
previousArgumentsIndex = 0
while True:
changedSomething = False
arguments = shlex.split(command)
for argumentsIndex in range(previousArgumentsIndex, len(arguments) - 1):
argument, nextArgument = arguments[argumentsIndex : argumentsIndex + 2]
if argument == '-H':
previousCommand = command
printTryToRemove(arguments[argumentsIndex : argumentsIndex + 2])
del arguments[argumentsIndex : argumentsIndex + 2]
command = shlex.join(arguments)
if isCommandStillFine(command):
printThatCommandIsStillFine(command)
previousArgumentsIndex = argumentsIndex
changedSomething = True
break
else:
command = previousCommand
arguments = shlex.split(command)
if not changedSomething:
break
if removeUrlParameters:
print('Removing URL parameters')
arguments = shlex.split(command)
for argumentsIndex, argument in enumerate(arguments):
if re.match('https?://', argument):
urlIndex = argumentsIndex
break
def getUrl(urlParsed, query):
return urlParsed._replace(query = '&'.join([f'{quote_plus(parameter)}={quote_plus(query[parameter][0])}' for parameter in query])).geturl()
url = arguments[urlIndex]
previousKeyIndex = 0
while True:
changedSomething = False
urlParsed = urlparse(url)
query = parse_qs(urlParsed.query, keep_blank_values = True)
for keyIndex, key in enumerate(list(query)[previousKeyIndex:]):
previousQuery = copy.deepcopy(query)
printTryToRemove(key)
del query[key]
# Make a function with below code.
url = getUrl(urlParsed, query)
arguments[urlIndex] = url
command = shlex.join(arguments)
if isCommandStillFine(command):
printThatCommandIsStillFine(command)
changedSomething = True
previousKeyIndex = keyIndex
break
else:
query = previousQuery
url = getUrl(urlParsed, query)
arguments[urlIndex] = url
command = shlex.join(arguments)
if not changedSomething:
break
if removeCookies:
print('Removing cookies')
COOKIES_PREFIX = 'Cookie: '
COOKIES_PREFIX_LEN = len(COOKIES_PREFIX)
cookiesIndex = None
arguments = shlex.split(command)
for argumentsIndex, argument in enumerate(arguments):
# For Chromium support:
if argument[:COOKIES_PREFIX_LEN].title() == COOKIES_PREFIX:
cookiesIndex = argumentsIndex
arguments[cookiesIndex] = COOKIES_PREFIX + argument[COOKIES_PREFIX_LEN:]
break
if cookiesIndex is not None:
cookies = arguments[cookiesIndex]
previousCookiesParsedIndex = 0
while True:
changedSomething = False
cookiesParsed = cookies.replace(COOKIES_PREFIX, '').split('; ')
for cookiesParsedIndex, cookie in enumerate(cookiesParsed[previousCookiesParsedIndex:]):
cookiesParsedCopy = cookiesParsed[:]
printTryToRemove(cookie)
del cookiesParsedCopy[cookiesParsedIndex]
arguments[cookiesIndex] = COOKIES_PREFIX + '; '.join(cookiesParsedCopy)
command = shlex.join(arguments)
if isCommandStillFine(command):
printThatCommandIsStillFine(command)
changedSomething = True
previousCookiesParsedIndex = cookiesParsedIndex
cookies = '; '.join(cookiesParsedCopy)
break
else:
arguments[cookiesIndex] = COOKIES_PREFIX + '; '.join(cookiesParsed)
command = shlex.join(arguments)
if not changedSomething:
break
if removeRawData:
print('Removing raw data')
rawDataIndex = None
isJson = False
arguments = shlex.split(command)
for argumentsIndex, argument in enumerate(arguments):
if argumentsIndex > 0 and arguments[argumentsIndex - 1] == '--data-raw':
rawDataIndex = argumentsIndex
try:
json.loads(argument)
isJson = True
except:
pass
break
if rawDataIndex is not None:
rawData = arguments[rawDataIndex]
# Could interwine both cases but don't seem to clean much the code due to `getPaths` notably.
# Just firstly making a common function to all parts minimizer would make sense.
if not isJson:
previousRawDataPartsIndex = 0
while rawData != '':
changedSomething = False
rawDataParts = rawData.split('&')
for rawDataPartsIndex, rawDataPart in enumerate(rawDataParts[previousRawDataPartsIndex:]):
rawDataPartsCopy = copy.deepcopy(rawDataParts)
printTryToRemove(rawDataPartsCopy[rawDataPartsIndex])
del rawDataPartsCopy[rawDataPartsIndex]
arguments[rawDataIndex] = '&'.join(rawDataPartsCopy)
command = shlex.join(arguments)
if isCommandStillFine(command):
printThatCommandIsStillFine(command)
changedSomething = True
previousRawDataPartsIndex = rawDataPartsIndex
rawData = '&'.join(rawDataPartsCopy)
break
else:
arguments[rawDataIndex] = '&'.join(rawDataParts)
command = shlex.join(arguments)
if not changedSomething:
break
# JSON recursive case.
else:
def getPaths(d):
if isinstance(d, dict):
for key, value in d.items():
yield f'/{key}'
yield from (f'/{key}{p}' for p in getPaths(value))
elif isinstance(d, list):
for i, value in enumerate(d):
yield f'/{i}'
yield from (f'/{i}{p}' for p in getPaths(value))
# If a single unknown entry is necessary, then this algorithm seems to most efficiently goes from parents to children if necessary to remove other entries. Hence, it seems to proceed in a linear number of HTTPS requests and not a quadratic one.
# Try until no more change to remove unnecessary entries. If assume a logical behavior as just mentioned, would not a single loop iteration be enough? Not with current design, see (1).
previousPathsIndex = 0
while True:
changedSomething = False
rawDataParsed = json.loads(rawData)
# Note that the path goes from parents to children if necessary which is quite a wanted behavior to quickly remove useless chunks.
paths = getPaths(rawDataParsed)
# For all entries, copy current `rawData` and try to remove an entry.
for pathsIndex, path in enumerate(list(paths)[previousPathsIndex:]):
# Copy current `rawData`.
rawDataParsedCopy = copy.deepcopy(rawDataParsed)
# Remove an entry.
# Pay attention that integer keys need to be consider as such, so not as `str` as face a `list` instead of a `dict`.
entry = rawDataParsedCopy
pathParts = path[1:].split('/')
for pathPart in pathParts[:-1]:
pathPart = pathPart if not pathPart.isdigit() else int(pathPart)
entry = entry[pathPart]
lastPathPart = pathParts[-1]
lastPathPart = lastPathPart if not lastPathPart.isdigit() else int(lastPathPart)
printTryToRemove(path)
del entry[lastPathPart]
# Test if the removed entry was necessary.
arguments[rawDataIndex] = json.dumps(rawDataParsedCopy)
command = shlex.join(arguments)
# (1) If it was unnecessary, then reconsider paths excluding possible children paths of this unnecessary entry, ensuring optimized complexity it seems.
if isCommandStillFine(command):
printThatCommandIsStillFine(command)
changedSomething = True
previousPathsIndex = pathsIndex
rawData = json.dumps(rawDataParsedCopy)
break
# If it was necessary, we consider possible children paths of this necessary entry and other paths.
else:
arguments[rawDataIndex] = json.dumps(rawDataParsed)
command = shlex.join(arguments)
# If a loop iteration considering all paths, does not change anything, then the request cannot be minimized further.
if not changedSomething:
break
command = command.replace(' --compressed', '')
command = command.replace(' --data-raw \'\'', '')
HTTP_METHOD = ' -X POST'
if HTTP_METHOD in command:
previousCommand = command
command = command.replace(HTTP_METHOD, '')
if not isCommandStillFine(command):
command = previousCommand
# First test `print`ing, before potentially removing `minimized_curl` writing.
print(command)
writeCommand('minimized_curl', command)
os.remove(getCommandScript(PARTIALLY_MINIMIZED_CURL_SCRIPT_NAME))
================================================
FILE: tools/simplifyCURL.py
================================================
#!/usr/bin/env python
# in fact this tool can be used not only for YouTube but for all automatization based on HTTP
import os
import subprocess
import re
import json
isWSL = True
def getPath(path):
return path if not isWSL else path.replace('\\', '/').replace('C:', '/mnt/c')
path = getPath('C:\\Users\\Benjamin\\Desktop\\BensFolder\\DEV\\StackOverflow\\YouTube\\')
os.chdir(path)
def exec(cmd):
return subprocess.check_output(cmd, shell=True).decode('utf-8')
with open('curlCommand.txt') as f:
line = f.readline()
needle = 'isHearted'
#print(line)
'''
two approaches:
1. block manually some useless pattern
2. automatically remove some patterns while keeping needle retrieved
'''
# beautify: echo '{JSON}' | python -m json.tool
headersToRemove = ['Accept-Encoding', 'User-Agent', 'Accept', 'Accept-Language', 'X-Goog-Visitor-Id', 'Sec-Fetch-Dest', 'DNT', 'Connection', 'Origin', 'X-Youtube-Client-Version', 'X-Youtube-Client-Name', 'Cookie', 'Sec-Fetch-Mode', 'Sec-Fetch-Site', 'Pragma', 'Cache-Control', 'TE'] # likewise more general # 'Referer' required for youtube music
# or could make a whitelist instead
toRemoves = [' -X POST']
# could also make one big but doing like currently we give some semantical structure and can then for instance make bruteforce
toReplaces = [['curl', 'curl -s'], ['2.20220119.05.00', '2.2022011'], ['1.20220125.01.00', '1.2022012'], ['%3D', '=']] # 2.20220201.05.00 -> 2.20220201
#dataRawNeedle = ' --data-raw \''
contextToRemoves = ['adSignalsInfo', 'user', 'request', 'clickTracking', 'clientScreenNonce']
clientToRemoves = ['hl', 'gl', 'remoteHost', 'deviceMake', 'deviceModel', 'userAgent', 'osName', 'osVersion', 'originalUrl', 'platform', 'clientFormFactor', 'configInfo', 'browserName', 'browserVersion', 'visitorData', 'screenWidthPoints', 'screenHeightPoints', 'screenPixelDensity', 'screenDensityFloat', 'utcOffsetMinutes', 'userInterfaceTheme', 'mainAppWebInfo', 'timeZone', 'playerType', 'tvAppInfo', 'clientScreen']
generalToRemoves = ['webSearchboxStatsUrl', 'playbackContext', 'cpn', 'captionParams', 'playlistId']
def delete(variable, sub):
if sub in variable:
del(variable[sub])
def treat(line):
for headerToRemove in headersToRemove:
#line = re.sub(r"-H '" + headerToRemove + ": [^']'", '', line)
line = re.sub(f' -H\s+\'{headerToRemove}(.+?)\'', '', line) # can starts with r"XYZ"
for toRemove in toRemoves:
line = line.replace(toRemove, '')
for toReplace in toReplaces:
needle, replaceWith = toReplace
line = line.replace(needle, replaceWith)
#if dataRawNeedle in line:
regex = '--data-raw\s+\'(.+?)\''
search = re.search(regex, line)
if search:
dataRaw = search.group(1)
#print(dataRaw)
#lineParts = line.split(dataRawNeedle)
#linePartsParts = lineParts[1].split('\'')
#dataRaw = linePartsParts[0] # could also use a regex
dataRawJson = json.loads(dataRaw)
for contextToRemove in contextToRemoves:
delete(dataRawJson['context'], contextToRemove)
for clientToRemove in clientToRemoves:
delete(dataRawJson['context']['client'], clientToRemove) # could generalize with n arguments with ... notation
#del(dataRawJson['webSearchboxStatsUrl'])
for generalToRemove in generalToRemoves:
delete(dataRawJson, generalToRemove)
newDataRaw = json.dumps(dataRawJson, separators=(',', ':'))
#print(json.dumps(dataRawJson, separators=(',', ':'), indent = 4))
line = re.sub(regex, '--data-raw \'{newDataRaw}\'', line)
#line = lineParts[0] + dataRawNeedle + newDataRaw + '\''# + linePartsParts[1]
return line
cmd = treat(line)
print(cmd)
res = exec(cmd)
if needle in res:
print('working')
else:
print('not working:')
print(res)
#print(res)
================================================
FILE: videos.php
================================================
<?php
header('Content-Type: application/json; charset=UTF-8');
include_once 'common.php';
$videosTests = [
['part=id&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs', 'items/0/videoId', 'NiXD4xVJM5Y'],
['part=clip&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs', 'items/0/clip', json_decode(file_get_contents('tests/part=clip&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs.json'), true)],
['part=contentDetails&id=g5xNzUA5Qf8', 'items/0/contentDetails/duration', 213],
['part=status&id=J8ZVxDK11Jo', 'items/0/status/embeddable', false],
['part=status&id=g5xNzUA5Qf8', 'items/0/status/embeddable', true], // could allow subarray for JSON check in response that way in a single request can check several features
['part=short&id=NiXD4xVJM5Y', 'items/0/short/available', false],
['part=short&id=ydPkyvWtmg4', 'items/0/short/available', true],
['part=musics&id=DUT5rEU6pqM', 'items/0/musics/0', json_decode(file_get_contents('tests/part=musics&id=DUT5rEU6pqM.json'), true)],
['part=musics&id=4sC3mbkP_x8', 'items/0/musics', json_decode(file_get_contents('tests/part=musics&id=4sC3mbkP_x8.json'), true)],
['part=music&id=FliCdfxdtTI', 'items/0/music/available', false],
['part=music&id=ntG3GQdY_Ok', 'items/0/music/available', true],
['part=isPaidPromotion&id=Q6gtj1ynstU', 'items/0/isPaidPromotion', false],
['part=isPaidPromotion&id=PEorJqo2Qaw', 'items/0/isPaidPromotion', true],
['part=isMemberOnly&id=Q6gtj1ynstU', 'items/0/isMemberOnly', false],
['part=isMemberOnly&id=Ln9yZDtfcWg', 'items/0/isMemberOnly', true],
['part=qualities&id=IkXH9H2ofa0', 'items/0/qualities', json_decode(file_get_contents('tests/part=qualities&id=IkXH9H2ofa0.json'), true)],
['part=chapters&id=n8vmXvoVjZw', 'items/0/chapters', json_decode(file_get_contents('tests/part=chapters&id=n8vmXvoVjZw.json'), true)],
['part=isOriginal&id=FliCdfxdtTI', 'items/0/isOriginal', false],
['part=isOriginal&id=iqKdEhx-dD4', 'items/0/isOriginal', true],
['part=isPremium&id=FliCdfxdtTI', 'items/0/isPremium', false],
['part=isPremium&id=dNJMI92NZJ0', 'items/0/isPremium', true],
['part=isRestricted&id=IkXH9H2ofa0', 'items/0/isRestricted', false],
['part=isRestricted&id=ORdWE_ffirg', 'items/0/isRestricted', true],
['part=snippet&id=IkXH9H2ofa0', 'items/0/snippet', json_decode(file_get_contents('tests/part=snippet&id=IkXH9H2ofa0.json'), true)],
['part=activity&id=V6z0qF54RZ4', 'items/0/activity', json_decode(file_get_contents('tests/part=activity&id=V6z0qF54RZ4.json'), true)],
['part=mostReplayed&id=XiCrniLQGYc', 'items/0/mostReplayed/markers/0/intensityScoreNormalized', 1],
['part=explicitLyrics&id=Ehoe35hTbuY', 'items/0/explicitLyrics', false],
['part=explicitLyrics&id=PvM79DJ2PmM', 'items/0/explicitLyrics', true],
];
$realOptions = [
'id',
'status',
'contentDetails',
'music',
'short',
'impressions',
'musics',
'isPaidPromotion',
'isPremium',
'isMemberOnly',
'mostReplayed',
'qualities',
'captions',
'location',
'chapters',
'isOriginal',
'isRestricted',
'snippet',
'clip',
'activity',
'explicitLyrics',
'statistics',
];
// really necessary ?
foreach ($realOptions as $realOption) {
$options[$realOption] = false;
}
if (isset($_GET['part']) && (isset($_GET['id']) || isset($_GET['clipId']))) {
$part = $_GET['part'];
$parts = explode(',', $part, count($realOptions));
foreach ($parts as $part) {
if (!in_array($part, $realOptions)) {
dieWithJsonMessage("Invalid part $part");
} else {
$options[$part] = true;
}
}
$isClip = isset($_GET['clipId']);
$field = $isClip ? 'clipId' : 'id';
$realIds = getMultipleIds($field);
foreach ($realIds as $realId) {
if ((!$isClip && !isVideoId($realId)) && !isClipId($realId)) {
dieWithJsonMessage("Invalid $field");
}
}
if ($options['impressions'] && (!isset($_GET['SAPISIDHASH']) || !isSAPISIDHASH($_GET['SAPISIDHASH']))) {
dieWithJsonMessage('Invalid SAPISIDHASH');
}
echo getAPI($realIds);
} else if(!test()) {
dieWithJsonMessage('Required parameters not provided');
}
function getJSONFunc($rawData, $music = false)
{
$headers = [
'Content-Type: application/json',
'Accept-Language: en'
];
if ($music) {
array_push($headers, 'Referer: https://music.youtube.com');
}
$opts = [
'http' => [
'method' => 'POST',
'header' => $headers,
'content' => $rawData,
]
];
return getJSON('https://' . ($music ? 'music' : 'www') . '.youtube.com/youtubei/v1/player?key=' . UI_KEY, $opts);
}
function getItem($id)
{
global $options;
$result = '';
if ($options['status'] || $options['contentDetails']) {
$rawData = [
'videoId' => $id,
'context' => [
'client' => [
'clientName' => 'WEB_EMBEDDED_PLAYER',
'clientVersion' => CLIENT_VERSION
]
]
];
$result = getJSONFunc(json_encode($rawData));
}
$item = [
'kind' => 'youtube#video',
'etag' => 'NotImplemented',
'id' => $id
];
if ($options['status']) {
$status = [
'embeddable' => $result['playabilityStatus']['status'] === 'OK',
'removedByTheUploader' => $result['playabilityStatus']['errorScreen']['playerErrorMessageRenderer']['subreason']['runs'][0]['text'] === 'This video has been removed by the uploader'
];
$item['status'] = $status;
}
if ($options['contentDetails']) {
$contentDetails = [
'duration' => intval($result['videoDetails']['lengthSeconds'])
];
$item['contentDetails'] = $contentDetails;
}
if ($options['music']) {
// music request doesn't provide embeddable info - could not make a request if only music and contentDetails
$rawData = [
'videoId' => $id,
'context' => [
'client' => [
'clientName' => 'WEB_REMIX',
'clientVersion' => CLIENT_VERSION
]
]
];
$resultMusic = getJSONFunc(json_encode($rawData), true);
$music = [
'available' => $resultMusic['playabilityStatus']['status'] === 'OK'
];
$item['music'] = $music;
}
if ($options['short']) {
$short = [
'available' => !isRedirection("https://www.youtube.com/shorts/$id")
];
$item['short'] = $short;
}
if ($options['impressions']) {
$headers = [
'x-origin: https://studio.youtube.com',
"authorization: SAPISIDHASH {$_GET['SAPISIDHASH']}",
'Content-Type:',
'Cookie: HSID=A4BqSu4moNA0Be1N9; SSID=AA0tycmNyGWo-Z_5v; APISID=a; SAPISID=zRbK-_14V7wIAieP/Ab_wY1sjLVrKQUM2c; SID=HwhYm6rJKOn_3R9oOrTNDJjpHIiq9Uos0F5fv4LPdMRSqyVHA1EDZwbLXo0kuUYAIN_MUQ.'
];
$rawData = [
'screenConfig' => [
'entity' => [
'videoId' => $id
]
],
'desktopState' => [
'tabId' => 'ANALYTICS_TAB_ID_REACH'
]
];
$opts = [
'http' => [
'method' => 'POST',
'header' => $headers,
'content' => json_encode($rawData),
]
];
$json = getJSON('https://studio.youtube.com/youtubei/v1/analytics_data/get_screen?key=' . UI_KEY, $opts);
$impressions = $json['cards'][0]['keyMetricCardData']['keyMetricTabs'][0]['primaryContent']['total'];
$item['impressions'] = $impressions;
}
if ($options['musics']) {
$opts = [
'http' => [
'header' => [
'Accept-Language: en',
]
]
];
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", $opts);
$musics = [];
$engagementPanels = $json['engagementPanels'];
$cardsPath = 'engagementPanelSectionListRenderer/content/structuredDescriptionContentRenderer/items';
$engagementPanel = getFirstNodeContainingPath($engagementPanels, $cardsPath);
$cards = getValue($engagementPanel, $cardsPath);
$cardsPath = 'horizontalCardListRenderer/cards';
$structuredDescriptionContentRendererItem = getFirstNodeContainingPath($cards, $cardsPath);
$cards = getValue($structuredDescriptionContentRendererItem, $cardsPath);
foreach ($cards as $card) {
$videoAttributeViewModel = $card['videoAttributeViewModel'];
$music = [
'image' => $videoAttributeViewModel['image']['sources'][0]['url'],
'videoId' => $videoAttributeViewModel['onTap']['innertubeCommand']['watchEndpoint']['videoId'],
];
$runs = $videoAttributeViewModel['overflowMenuOnTap']['innertubeCommand']['confirmDialogEndpoint']['content']['confirmDialogRenderer']['dialogMessages'][0]['runs'];
for($runIndex = 0; $runIndex < count($runs); $runIndex += 4)
{
$field = strtolower($runs[$runIndex]['text']);
$value = $runs[$runIndex + 2]['text'];
$music[$field] = $value;
}
array_push($musics, $music);
}
$item['musics'] = $musics;
}
if(isset($_GET['clipId'])) {
$json = getJSONFromHTML("https://www.youtube.com/clip/$id", forceLanguage: true);
if ($options['id']) {
$videoId = $json['currentVideoEndpoint']['watchEndpoint']['videoId'];
$item['videoId'] = $videoId;
}
if ($options['clip']) {
$engagementPanels = $json['engagementPanels'];
$path = 'engagementPanelSectionListRenderer/onShowCommands/0/showEngagementPanelScrimAction/onClickCommands/0/commandExecutorCommand/commands/3/openPopupAction/popup/notificationActionRenderer/actionButton/buttonRenderer/command/commandExecutorCommand/commands/1/loopCommand';
foreach ($engagementPanels as $engagementPanel) {
if (doesPathExist($engagementPanel, $path)) {
$loopCommand = getValue($engagementPanel, $path);
$clipAttributionRenderer = $engagementPanel['engagementPanelSectionListRenderer']['content']['clipSectionRenderer']['contents'][0]['clipAttributionRenderer'];
$createdText = explode(' · ', $clipAttributionRenderer['createdText']['simpleText']);
$clip = [
'title' => $engagementPanel['engagementPanelSectionListRenderer']['content']['clipSectionRenderer']['contents'][0]['clipAttributionRenderer']['title']['runs'][0]['text'],
'startTimeMs' => intval($loopCommand['startTimeMs']),
'endTimeMs' => intval($loopCommand['endTimeMs']),
'viewCount' => getIntValue($createdText[0], 'view'),
'publishedAt' => $createdText[1],
];
$item['clip'] = $clip;
break;
}
}
}
}
if ($options['isPaidPromotion']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", scriptVariable: 'ytInitialPlayerResponse');
$isPaidPromotion = array_key_exists('paidContentOverlay', $json);
$item['isPaidPromotion'] = $isPaidPromotion;
}
if ($options['isPremium']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id");
$isPremium = array_key_exists('offerModule', $json['contents']['twoColumnWatchNextResults']['secondaryResults']['secondaryResults']);
$item['isPremium'] = $isPremium;
}
if ($options['isMemberOnly']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", $opts);
$isMemberOnly = array_key_exists('badges', $json['contents']['twoColumnWatchNextResults']['results']['results']['contents'][0]['videoPrimaryInfoRenderer']);
$item['isMemberOnly'] = $isMemberOnly;
}
if ($options['mostReplayed']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", forceLanguage: true);
$mutations = $json['frameworkUpdates']['entityBatchUpdate']['mutations'];
$commonJsonPath = 'payload/macroMarkersListEntity/markersList';
$jsonPath = "$commonJsonPath/markersDecoration";
foreach($mutations as $mutation)
{
if(doesPathExist($mutation, $jsonPath))
{
break;
}
}
if(doesPathExist($mutation, $jsonPath))
{
$mostReplayed = getValue($mutation, $commonJsonPath);
foreach(array_keys($mostReplayed['markers']) as $markerIndex)
{
unset($mostReplayed['markers'][$markerIndex]['durationMillis']);
$mostReplayed['markers'][$markerIndex]['startMillis'] = intval($mostReplayed['markers'][$markerIndex]['startMillis']);
}
$timedMarkerDecorations = $mostReplayed['markersDecoration']['timedMarkerDecorations'];
foreach(array_keys($timedMarkerDecorations) as $timedMarkerDecorationIndex)
{
foreach(['label', 'icon', 'decorationTimeMillis'] as $timedMarkerDecorationKey)
{
unset($timedMarkerDecorations[$timedMarkerDecorationIndex][$timedMarkerDecorationKey]);
}
}
$mostReplayed['timedMarkerDecorations'] = $timedMarkerDecorations;
foreach(['markerType', 'markersMetadata', 'markersDecoration'] as $mostReplayedKey)
{
unset($mostReplayed[$mostReplayedKey]);
}
}
else
{
$mostReplayed = null;
}
$item['mostReplayed'] = $mostReplayed;
}
if ($options['qualities']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", scriptVariable: 'ytInitialPlayerResponse');
$qualities = [];
foreach ($json['streamingData']['adaptiveFormats'] as $quality) {
if (array_key_exists('qualityLabel', $quality)) {
$quality = $quality['qualityLabel'];
if (!in_array($quality, $qualities)) {
array_push($qualities, $quality);
}
}
}
$item['qualities'] = $qualities;
}
if ($options['captions']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", scriptVariable: 'ytInitialPlayerResponse', forceLanguage: true);
$captions = [];
foreach ($json['captions']['playerCaptionsTracklistRenderer']['captionTracks'] as $caption) {
array_push($captions, [
'name' => $caption['name']['simpleText'],
'languageCode' => $caption['languageCode'],
'kind' => $caption['kind'],
]);
}
$item['captions'] = $captions;
}
if ($options['location']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id");
$location = $json['contents']['twoColumnWatchNextResults']['results']['results']['contents'][0]['videoPrimaryInfoRenderer']['superTitleLink']['runs'][0]['text'];
$item['location'] = $location;
}
if ($options['chapters']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id");
$chapters = [];
$areAutoGenerated = false;
foreach ($json['engagementPanels'] as $engagementPanel) {
if ($engagementPanel['engagementPanelSectionListRenderer']['panelIdentifier'] === 'engagement-panel-macro-markers-description-chapters')
break;
}
$contents = $engagementPanel['engagementPanelSectionListRenderer']['content']['macroMarkersListRenderer']['contents'];
if ($contents !== null) {
$areAutoGenerated = array_key_exists('macroMarkersInfoItemRenderer', $contents[0]);
$contents = array_slice($contents, $areAutoGenerated ? 1 : 0);
foreach ($contents as $chapter) {
$chapter = $chapter['macroMarkersListItemRenderer'];
$timeInt = getIntFromDuration($chapter['timeDescription']['simpleText']);
array_push($chapters, [
'title' => $chapter['title']['simpleText'],
'time' => $timeInt,
'thumbnails' => $chapter['thumbnail']['thumbnails']
]);
}
}
$chapters = [
'areAutoGenerated' => $areAutoGenerated,
'chapters' => $chapters
];
$item['chapters'] = $chapters;
}
if ($options['isOriginal']) {
$json = getJson("https://www.youtube.com/watch?v=$id");
$isOriginal = doesPathExist($json, 'contents/twoColumnWatchNextResults/results/results/contents/1/videoSecondaryInfoRenderer/metadataRowContainer/metadataRowContainerRenderer/rows/2/metadataRowRenderer/contents/0/simpleText') or str_contains($html, 'xtags=' . urlencode('acont=original'));
$item['isOriginal'] = $isOriginal;
}
if ($options['isRestricted']) {
$opts = [
'http' => [
'header' => ['Cookie: PREF=f2=8000000'],
]
];
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", $opts, 'ytInitialPlayerResponse');
$playabilityStatus = $json['playabilityStatus'];
$isRestricted = array_key_exists('isBlockedInRestrictedMode', $playabilityStatus);
$item['isRestricted'] = $isRestricted;
}
if ($options['snippet']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", forceLanguage: true);
$contents = $json['contents']['twoColumnWatchNextResults']['results']['results']['contents'];
// Note that `publishedAt` has a day only precision.
$publishedAt = strtotime($contents[0]['videoPrimaryInfoRenderer']['dateText']['simpleText']);
$description = $contents[1]['videoSecondaryInfoRenderer']['attributedDescription']['content'];
$snippet = [
'publishedAt' => $publishedAt,
'description' => $description,
'title' => $contents[0]['videoPrimaryInfoRenderer']['title']['runs'][0]['text'],
];
$item['snippet'] = $snippet;
}
if ($options['statistics']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", forceLanguage: true);
preg_match('/like this video along with ([\d,]+) other people/', $json['contents']['twoColumnWatchNextResults']['results']['results']['contents'][0]['videoPrimaryInfoRenderer']['videoActions']['menuRenderer']['topLevelButtons'][0]['segmentedLikeDislikeButtonViewModel']['likeButtonViewModel']['likeButtonViewModel']['toggleButtonViewModel']['toggleButtonViewModel']['defaultButtonViewModel']['buttonViewModel']['accessibilityText'], $viewCount);
$statistics = [
'viewCount' => getIntValue($json['playerOverlays']['playerOverlayRenderer']['videoDetails']['playerOverlayVideoDetailsRenderer']['subtitle']['runs'][2]['text'], 'view'),
'likeCount' => getIntValue($viewCount[1]),
];
$item['statistics'] = $statistics;
}
if ($options['activity']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", forceLanguage: true);
$activity = $json['contents']['twoColumnWatchNextResults']['results']['results']['contents'][1]['videoSecondaryInfoRenderer']['metadataRowContainer']['metadataRowContainerRenderer']['rows'][0]['richMetadataRowRenderer']['contents'][0]['richMetadataRenderer'];
$name = $activity['title']['simpleText'];
$year = $activity['subtitle']['simpleText'];
$thumbnails = $activity['thumbnail']['thumbnails'];
$channelId = $activity['endpoint']['browseEndpoint']['browseId'];
$activity = [
'name' => $name,
'year' => $year,
'thumbnails' => $thumbnails,
'channelId' => $channelId
];
$item['activity'] = $activity;
}
if ($options['explicitLyrics']) {
$json = getJSONFromHTML("https://www.youtube.com/watch?v=$id");
$rows = $json['contents']['twoColumnWatchNextResults']['results']['results']['contents'][1]['videoSecondaryInfoRenderer']['metadataRowContainer']['metadataRowContainerRenderer']['rows'];
$item['explicitLyrics'] = $rows !== null && end($rows)['metadataRowRenderer']['contents'][0]['simpleText'] === 'Explicit lyrics';
}
return $item;
}
function getAPI($ids)
{
$items = [];
foreach ($ids as $id) {
array_push($items, getItem($id));
}
$answer = [
'kind' => 'youtube#videoListResponse',
'etag' => 'NotImplemented',
'items' => $items
];
return json_encode($answer, JSON_PRETTY_PRINT);
}
================================================
FILE: ytPrivate/.htaccess
================================================
<FilesMatch "keys.txt">
Deny from all
</FilesMatch>
================================================
FILE: ytPrivate/test.php
================================================
<?php
function array_intersect_recursive(&$value1, &$value2)
{
$intersectKeys = array_intersect(array_keys($value1), array_keys($value2));
$intersectValues = [];
foreach ($intersectKeys as $key) {
$element1 = $value1[$key];
$element2 = $value2[$key];
if(is_array($element1) && is_array($element2))
{
$intersectValues[$key] = array_intersect_recursive($element1, $element2);
}
else if(!is_array($element1) && !is_array($element2) && $element1 === $element2)
{
$intersectValues[$key] = $element1;
}
}
return $intersectValues;
}
$endpoint = $argv[1];
// Used in endpoint.
$test = true;
require_once "../$endpoint.php";
$tests = $GLOBALS["{$endpoint}Tests"];
foreach ($tests as $test) {
$url = $test[0];
$jsonPath = $test[1];
$groundTruthValue = $test[2];
// Should not use network but call the PHP files thanks to `include` etc and provide arguments correctly instead.
$retrievedContent = shell_exec("php-cgi ../$endpoint.php " . escapeshellarg($url));
$retrievedContent = str_replace('Content-Type: application/json; charset=UTF-8', '', $retrievedContent);
$retrievedContentJson = json_decode($retrievedContent, true);
$jsonPathExistsInRetrievedContentJson = doesPathExist($retrievedContentJson, $jsonPath);
$retrievedContentValue = $jsonPathExistsInRetrievedContentJson ? getValue($retrievedContentJson, $jsonPath) : '';
$isGroundTruthValueAnArrayAndEqualToRetrievedContentValue = is_array($groundTruthValue) && array_intersect_recursive($retrievedContentValue, $groundTruthValue) == $groundTruthValue;
$isGroundTruthValueNotAnArrayAndEqualToRetrievedContentValue = !is_array($groundTruthValue) && $retrievedContentValue === $groundTruthValue;
$valueInclusion = $isGroundTruthValueAnArrayAndEqualToRetrievedContentValue || $isGroundTruthValueNotAnArrayAndEqualToRetrievedContentValue;
$testSuccessful = $jsonPathExistsInRetrievedContentJson && $valueInclusion;
$groundTruthValue = is_array($groundTruthValue) ? 'Array' : $groundTruthValue;
$retrievedContentValue = is_array($retrievedContentValue) ? 'Array' : $retrievedContentValue;
echo($testSuccessful ? 'PASS' : 'FAIL') . " $endpoint $url $jsonPath $groundTruthValue" . ($testSuccessful ? '' : " $retrievedContentValue") . "\n";
}
================================================
FILE: ytPrivate/tests.php
================================================
<?php
$endpoints = [
'channels',
'commentThreads',
'community',
'liveChats',
'lives',
'playlistItems',
'playlists',
'search',
'videos',
];
foreach ($endpoints as $endpoint) {
system("php test.php $endpoint");
} // that way don't have to reset context from a test to the other
// deepen some tests would be nice
gitextract_ugyzn8kf/
├── .gitignore
├── .htaccess
├── CITATION.cff
├── CONTRIBUTING.md
├── Dockerfile
├── README.md
├── addKey.php
├── channels.php
├── commentThreads.php
├── common.php
├── community.php
├── configuration.php
├── constants.php
├── docker-compose.yml
├── index.php
├── keys.php
├── liveChats.php
├── lives.php
├── noKey/
│ ├── .htaccess
│ └── index.php
├── playlistItems.php
├── playlists.php
├── proto/
│ ├── php/
│ │ └── .gitignore
│ └── prototypes/
│ ├── browse.proto
│ └── browse_shorts.proto
├── search.php
├── tools/
│ ├── checkOperationnalAPI.py
│ ├── getJSONPathFromKey.py
│ ├── minimizeCURL.py
│ └── simplifyCURL.py
├── videos.php
└── ytPrivate/
├── .htaccess
├── test.php
└── tests.php
SYMBOL INDEX (94 symbols across 17 files)
FILE: channels.php
function getItem (line 142) | function getItem($id, $order, $continuationToken)
function returnItems (line 555) | function returnItems($items)
function getAPI (line 567) | function getAPI($ids, $order, $continuationToken)
function getVisitorData (line 576) | function getVisitorData($result)
function getVideo (line 581) | function getVideo($gridRendererItem)
function getVideos (line 602) | function getVideos(&$item, $url, $getGridRendererItems, $continuationToken)
function getVideoFromItsThumbnails (line 629) | function getVideoFromItsThumbnails($videoThumbnails, $isVideo = true) {
function getFirstVideos (line 640) | function getFirstVideos($playlistRenderer)
FILE: commentThreads.php
function getAPI (line 71) | function getAPI($videoId, $commentId, $order, $continuationToken, $simul...
FILE: common.php
function getContextFromOpts (line 23) | function getContextFromOpts($opts)
function getHeadersFromOpts (line 59) | function getHeadersFromOpts($url, $opts)
function fileGetContentsAndHeadersFromOpts (line 66) | function fileGetContentsAndHeadersFromOpts($url, $opts)
function isRedirection (line 88) | function isRedirection($url)
function getRemote (line 104) | function getRemote($url, $opts = [], $verifyTrafficIfForbidden = true)
function dieWithJsonMessage (line 115) | function dieWithJsonMessage($message, $code = 400)
function detectedAsSendingUnusualTraffic (line 127) | function detectedAsSendingUnusualTraffic()
function getJSON (line 132) | function getJSON($url, $opts = [], $verifyTrafficIfForbidden = true)
function getJSONStringFromHTMLScriptPrefix (line 137) | function getJSONStringFromHTMLScriptPrefix($html, $scriptPrefix)
function getJSONFromHTMLScriptPrefix (line 143) | function getJSONFromHTMLScriptPrefix($html, $scriptPrefix)
function getJSONStringFromHTML (line 149) | function getJSONStringFromHTML($html, $scriptVariable = '', $prefix = 'v...
function getJSONFromHTML (line 158) | function getJSONFromHTML($url, $opts = [], $scriptVariable = '', $prefix...
function checkRegex (line 186) | function checkRegex($regex, $str)
function isContinuationToken (line 191) | function isContinuationToken($continuationToken)
function isContinuationTokenAndVisitorData (line 196) | function isContinuationTokenAndVisitorData($continuationTokenAndVisitorD...
function isPlaylistId (line 201) | function isPlaylistId($playlistId)
function isCId (line 208) | function isCId($cId)
function isUsername (line 213) | function isUsername($username)
function isChannelId (line 218) | function isChannelId($channelId)
function isVideoId (line 223) | function isVideoId($videoId)
function isHashtag (line 228) | function isHashtag($hashtag)
function isSAPISIDHASH (line 233) | function isSAPISIDHASH($SAPISIDHASH)
function isQuery (line 238) | function isQuery($q)
function isClipId (line 243) | function isClipId($clipId)
function isEventType (line 248) | function isEventType($eventType)
function isPositiveInteger (line 253) | function isPositiveInteger($s)
function isYouTubeDataAPIV3Key (line 258) | function isYouTubeDataAPIV3Key($youtubeDataAPIV3Key)
function isHandle (line 263) | function isHandle($handle)
function isPostId (line 268) | function isPostId($postId)
function isCommentId (line 273) | function isCommentId($commentId)
function doesPathExist (line 279) | function doesPathExist($json, $path)
function getValue (line 292) | function getValue($json, $path, $defaultPath = null, $defaultValue = null)
function getIntValue (line 308) | function getIntValue($unitCount, $unit = '')
function getCommunityPostFromContent (line 325) | function getCommunityPostFromContent($content)
function getIntFromViewCount (line 422) | function getIntFromViewCount($viewCount)
function getIntFromDuration (line 435) | function getIntFromDuration($timeStr)
function getFirstNodeContainingPath (line 461) | function getFirstNodeContainingPath($nodes, $path) {
function getTabByName (line 465) | function getTabByName($result, $tabName) {
function getPublishedAt (line 473) | function getPublishedAt($publishedAtRaw) {
function test (line 507) | function test()
function getContinuationItems (line 513) | function getContinuationItems($result)
function getTabs (line 518) | function getTabs($result)
function getContinuationJson (line 523) | function getContinuationJson($continuationToken)
function verifyMultipleIdsConfiguration (line 560) | function verifyMultipleIdsConfiguration($realIds, $field) {
function verifyTooManyIds (line 566) | function verifyTooManyIds($realIds, $field) {
function verifyMultipleIds (line 572) | function verifyMultipleIds($realIds, $field = 'id') {
function getMultipleIds (line 577) | function getMultipleIds($field) {
function includeOnceProto (line 584) | function includeOnceProto($proto) {
function includeOnceProtos (line 590) | function includeOnceProtos($protos) {
function base64url_encode (line 598) | function base64url_encode($data) {
FILE: community.php
function implodeArray (line 48) | function implodeArray($anArray, $separator)
function getAPI (line 53) | function getAPI($postId, $channelId, $order)
FILE: index.php
function url (line 41) | function url($url, $name = '')
function yt (line 49) | function yt()
function getUrl (line 54) | function getUrl($parameters)
function feature (line 59) | function feature($feature)
FILE: liveChats.php
function getItem (line 47) | function getItem($id)
function getAPI (line 121) | function getAPI($ids)
FILE: lives.php
function getItem (line 45) | function getItem($id)
function getAPI (line 156) | function getAPI($ids)
FILE: noKey/index.php
function myDie (line 23) | function myDie($content)
FILE: playlistItems.php
function getAPI (line 34) | function getAPI($playlistId, $continuationToken)
FILE: playlists.php
function getItem (line 45) | function getItem($id)
function getAPI (line 75) | function getAPI($ids)
FILE: search.php
function getAPI (line 101) | function getAPI($id, $order, $continuationToken)
FILE: tools/checkOperationnalAPI.py
function getVideos (line 9) | def getVideos(pageToken = '', callsIndex = 0):
FILE: tools/getJSONPathFromKey.py
function treatKey (line 31) | def treatKey(obj, path, key):
function _finditem (line 38) | def _finditem(obj, key, path = ''):
FILE: tools/minimizeCURL.py
function printTryToRemove (line 42) | def printTryToRemove(toRemove):
function executeCommand (line 50) | def executeCommand(command):
function getCommandScript (line 59) | def getCommandScript(fileName):
function writeCommand (line 62) | def writeCommand(fileName, command):
function isCommandStillFine (line 68) | def isCommandStillFine(command):
function getCommandLengthFormatted (line 76) | def getCommandLengthFormatted(command):
function printThatCommandIsStillFine (line 79) | def printThatCommandIsStillFine(command):
function getUrl (line 126) | def getUrl(urlParsed, query):
function getPaths (line 239) | def getPaths(d):
FILE: tools/simplifyCURL.py
function getPath (line 12) | def getPath(path):
function exec (line 19) | def exec(cmd):
function delete (line 50) | def delete(variable, sub):
function treat (line 54) | def treat(line):
FILE: videos.php
function getJSONFunc (line 96) | function getJSONFunc($rawData, $music = false)
function getItem (line 115) | function getItem($id)
function getAPI (line 465) | function getAPI($ids)
FILE: ytPrivate/test.php
function array_intersect_recursive (line 3) | function array_intersect_recursive(&$value1, &$value2)
Condensed preview — 34 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (182K chars).
[
{
"path": ".gitignore",
"chars": 24,
"preview": ".env\nytPrivate/keys.txt\n"
},
{
"path": ".htaccess",
"chars": 955,
"preview": "Options +FollowSymLinks\nRewriteEngine on\n\nRewriteRule ^search$ search.php\nRewriteRule ^videos$ videos.php\nRewriteRule ^p"
},
{
"path": "CITATION.cff",
"chars": 496,
"preview": "cff-version: 1.2.0\ntitle: YouTube operational API\nmessage: >-\n If you use this software, please cite it using the\n met"
},
{
"path": "CONTRIBUTING.md",
"chars": 993,
"preview": "# Welcome to the YouTube operational API contributing guide\n\nThank you for investing your time in contributing to YouTub"
},
{
"path": "Dockerfile",
"chars": 647,
"preview": "FROM php:apache\n\nRUN a2enmod rewrite\n\n# Copy application files into the container\nCOPY . /var/www/html/\n\n# Replace `Allo"
},
{
"path": "README.md",
"chars": 4220,
"preview": "# YouTube operational API\nYouTube operational API works when [YouTube Data API v3](https://developers.google.com/youtube"
},
{
"path": "addKey.php",
"chars": 2374,
"preview": "<?php\n\n include_once 'common.php';\n\n if (isset($_GET['key'])) {\n $key = $_GET['key'];\n // Regex-base"
},
{
"path": "channels.php",
"chars": 36180,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n include_once 'common.php';\n\n $channelsTests "
},
{
"path": "commentThreads.php",
"chars": 8443,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n // Stack Overflow source: https://stackoverflow"
},
{
"path": "common.php",
"chars": 22628,
"preview": "<?php\n\n include_once 'constants.php';\n include_once 'configuration.php';\n\n ini_set('display_errors', 0);\n\n i"
},
{
"path": "community.php",
"chars": 3891,
"preview": "<?php\n\nheader('Content-Type: application/json; charset=UTF-8');\n\ninclude_once 'common.php';\n\nincludeOnceProtos(['Browse'"
},
{
"path": "configuration.php",
"chars": 760,
"preview": "<?php\n\n // Global:\n define('SERVER_NAME', 'my instance');\n\n // Web-scraping endpoints:\n define('GOOGLE_ABUSE"
},
{
"path": "constants.php",
"chars": 982,
"preview": "<?php\r\n\r\n include_once 'configuration.php';\r\n\r\n $newAddKeyForceSecret = ADD_KEY_FORCE_SECRET;\r\n if (ADD_KEY_FOR"
},
{
"path": "docker-compose.yml",
"chars": 147,
"preview": "version: '3.8'\nservices:\n api:\n image: youtube-operational-api\n build: .\n restart: on-failure\n ports:\n "
},
{
"path": "index.php",
"chars": 9139,
"preview": "<!DOCTYPE html>\n<html lang=\"en\">\n <head>\n <meta charset=\"UTF-8\" />\n <title>YouTube operational API</tit"
},
{
"path": "keys.php",
"chars": 292,
"preview": "<!-- could add here stuff to add/remove key cf https://github.com/Benjamin-Loison/YouTube-operational-API/issues/18 -->\n"
},
{
"path": "liveChats.php",
"chars": 4769,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n $liveTests = [];\n\n include_once 'common.php'"
},
{
"path": "lives.php",
"chars": 6515,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n $liveTests = [];\n\n include_once 'common.php'"
},
{
"path": "noKey/.htaccess",
"chars": 69,
"preview": "Options +FollowSymLinks\nRewriteEngine on\n\nRewriteRule ^.*$ index.php\n"
},
{
"path": "noKey/index.php",
"chars": 4946,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n chdir('..');\n\n include_once 'common.php';\n\n "
},
{
"path": "playlistItems.php",
"chars": 3971,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n $playlistItemsTests = [\n // not precise."
},
{
"path": "playlists.php",
"chars": 2699,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n $playlistsTests = [\n ['part=snippet&id=P"
},
{
"path": "proto/php/.gitignore",
"chars": 71,
"preview": "# Ignore everything in this directory\n*\n# Except this file\n!.gitignore\n"
},
{
"path": "proto/prototypes/browse.proto",
"chars": 135,
"preview": "syntax = \"proto3\";\n\nmessage Browse {\n string endpoint = 2;\n SubBrowse subBrowse = 25;\n}\n\nmessage SubBrowse {\n string "
},
{
"path": "proto/prototypes/browse_shorts.proto",
"chars": 479,
"preview": "syntax = \"proto3\";\n\nmessage BrowseShorts {\n Sub0BrowseShorts two = 2;\n int32 three = 3;\n string four = 4;\n}\n\nmessage "
},
{
"path": "search.php",
"chars": 12714,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n // Stack Overflow source: https://stackoverflow"
},
{
"path": "tools/checkOperationnalAPI.py",
"chars": 586,
"preview": "#!/usr/bin/env python\n\nimport requests\n\nquery = 'test'\n\nvideoIds = []\n\ndef getVideos(pageToken = '', callsIndex = 0):\n "
},
{
"path": "tools/getJSONPathFromKey.py",
"chars": 4527,
"preview": "#!/usr/bin/env python\n\n'''\nThis script purpose is to ease retrieving the JSON path associated to an interested YouTube d"
},
{
"path": "tools/minimizeCURL.py",
"chars": 13028,
"preview": "#!/usr/bin/env python\n\n## /!\\ Assume that the content of `curlCommandFilePath` is trusted /!\\\n# TODO: precising or/and l"
},
{
"path": "tools/simplifyCURL.py",
"chars": 3855,
"preview": "#!/usr/bin/env python\n\n# in fact this tool can be used not only for YouTube but for all automatization based on HTTP\n\nim"
},
{
"path": "videos.php",
"chars": 22785,
"preview": "<?php\n\n header('Content-Type: application/json; charset=UTF-8');\n\n include_once 'common.php';\n\n $videosTests = "
},
{
"path": "ytPrivate/.htaccess",
"chars": 56,
"preview": "<FilesMatch \"keys.txt\">\n Deny from all\n</FilesMatch>\n"
},
{
"path": "ytPrivate/test.php",
"chars": 2453,
"preview": "<?php\n\nfunction array_intersect_recursive(&$value1, &$value2)\n{\n $intersectKeys = array_intersect(array_keys($value1)"
},
{
"path": "ytPrivate/tests.php",
"chars": 412,
"preview": "<?php\n\n $endpoints = [\n 'channels',\n 'commentThreads',\n 'community',\n 'liveChats',\n "
}
]
About this extraction
This page contains the full source code of the Benjamin-Loison/YouTube-operational-API GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 34 files (172.1 KB), approximately 41.9k tokens, and a symbol index with 94 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.