Hum. Problem.
Looks like we have a problem here.

You might want to return to the homepage.
And if you are already on the homepage, it might be a bigger problem. So don't move.
# Banditore

[](https://codecov.io/github/j0k3r/banditore)

Banditore retrieves new releases from your GitHub starred repositories and put them in a RSS feed, just for you.

## Requirements
- PHP >= 8.2 (with `pdo_mysql`)
- MySQL >= 5.7
- Redis (to cache requests to the GitHub API)
- [RabbitMQ](https://www.rabbitmq.com/), which is optional (see below)
- [Supervisor](http://supervisord.org/) (only if you use RabbitMQ)
## Installation
1. Clone the project
```bash
git clone https://github.com/j0k3r/banditore.git
```
2. [Register a new OAuth GitHub application](https://github.com/settings/applications/new) and get the _Client ID_ & _Client Secret_ for the next step (for the _Authorization callback URL_ put `http://127.0.0.1:8000/callback`)
3. Install dependencies using [Composer](https://getcomposer.org/download/) and define your parameter during the installation
```bash
APP_ENV=prod composer install -o --no-dev
```
If you want to use:
- **Sentry** to retrieve all errors, [register here](https://sentry.io/signup/) and get your dsn (in Project Settings > DSN).
4. Setup the database
```bash
php bin/console doctrine:database:create -e prod
php bin/console doctrine:schema:create -e prod
```
5. The application serves its frontend assets directly from `public/`, so no Node/Yarn install step is required (it's locked on `font-awesome@4.7.0` & `purecss@3.0.0`).
6. You can now launch the website:
```bash
php -S localhost:8000 -t public/
```
And access it at this address: `http://127.0.0.1:8000`
## Running the instance
Once the website is up, you now have to setup few things to retrieve new releases.
You have two choices:
- using crontab command (very simple and ok if you are alone)
- using RabbitMQ (might be better if you plan to have more than few persons but it's more complex) :call_me_hand:
### Without RabbitMQ
You just need to define these 2 cronjobs (replace all `/path/to/banditore` with real value):
```bash
# retrieve new release of each repo every 10 minutes
*/10 * * * * php /path/to/banditore/bin/console -e prod banditore:sync:versions >> /path/to/banditore/var/logs/command-sync-versions.log 2>&1
# sync starred repos of each user every 5 minutes
*/5 * * * * php /path/to/banditore/bin/console -e prod banditore:sync:starred-repos >> /path/banditore/to/var/logs/command-sync-repos.log 2>&1
```
### With RabbitMQ
1. You'll need to declare exchanges and queues. Replace `guest` by the user of your RabbitMQ instance (`guest` is the default one):
```bash
php bin/console messenger:setup-transports -vvv sync_starred_repos
php bin/console messenger:setup-transports -vvv sync_versions
```
2. You now have two queues and two exchanges defined:
- `banditore.sync_starred_repos`: will receive messages to sync starred repos of all users
- `banditore.sync_versions`: will receive message to retrieve new release for repos
3. Enable these 2 cronjobs which will periodically push messages in queues (replace all `/path/to/banditore` with real value):
```bash
# retrieve new release of each repo every 10 minutes
*/10 * * * * php /path/to/banditore/bin/console -e prod banditore:sync:versions --use_queue >> /path/to/banditore/var/logs/command-sync-versions.log 2>&1
# sync starred repos of each user every 5 minutes
*/5 * * * * php /path/to/banditore/bin/console -e prod banditore:sync:starred-repos --use_queue >> /path/banditore/to/var/logs/command-sync-repos.log 2>&1
```
4. Setup Supervisor using the [sample file](data/supervisor.conf) from the repo. You can copy/paste it into `/etc/supervisor/conf.d/` and adjust path. The default file will launch:
- 2 workers for sync starred repos
- 4 workers to fetch new releases
Once you've put the file in the supervisor conf repo, run `supervisorctl update && supervisorctl start all` (`update` will read your conf, `start all` will start all workers)
### Monitoring
There is a status page available at `/status`, it returns a json with some information about the freshness of fetched versions:
```json
{
"latest": {
"date": "2019-09-17 19:50:50.000000",
"timezone_type": 3,
"timezone": "Europe\/Berlin"
},
"diff": 1736,
"is_fresh": true
}
```
- `latest`: the latest created version as a DateTime
- `diff`: the difference between now and the latest created version (in seconds)
- `is_fresh`: indicate if everything is fine by comparing the `diff` above with the `status_minute_interval_before_alert` parameter
For example, I've setup a check on [updown.io](https://updown.io/r/P7qer) to check that status page and if the page contains `"is_fresh":true`. So I receive an alert when `is_fresh` is false: which means there is a potential issue on the server.
## Running the test suite
If you plan to contribute (you're awesome, I know that :v:), you'll need to install the project in a different way (for example, to retrieve dev packages):
```bash
git clone https://github.com/j0k3r/banditore.git
composer install -o
php bin/console doctrine:database:create -e=test
php bin/console doctrine:schema:create -e=test
php bin/console doctrine:fixtures:load --env=test -n
php bin/phpunit
```
Test environment defaults, including the database connection, are defined in `.env.test`.
## How it works
Ok, if you goes that deeper in the readme, it means you're a bit more than interested, I like that.
### Retrieving new release / tag
This is the complex part of the app. Here is a simplified solution to achieve it.
#### New release
It's not as easy as using the `/repos/:owner/:repo/releases` API endpoint to retrieve latest release for a given repo. Because not all repo owner use that feature (which is a shame in my case).
All information for a release are available on that endpoint:
- name of the tag (ie: v1.0.0)
- name of the release (ie: yay first release)
- published date
- description of the release
> Check a new release of that repo as example: https://api.github.com/repos/j0k3r/banditore/releases/5770680
#### New tag
Some owners also use tag which is a bit more complex to retrieve all information because a tag only contains information about the SHA-1 of the commit which was used to make the tag.
We only have these information:
- name of the tag (ie: v1.4.2)
- name of the release will be the name of the tag, in that case
> Check tag list of swarrot/SwarrotBundle as example: https://api.github.com/repos/swarrot/SwarrotBundle/tags
After retrieving the tag, we need to retrieve the commit to get these information:
- date of the commit
- message of the commit
> Check a commit from the previous tag list as example: https://api.github.com/repos/swarrot/SwarrotBundle/commits/84c7c57622e4666ae5706f33cd71842639b78755
### GitHub Client Discovery
This is the most important piece of the app. One thing that I ran though is hitting the rate limit on GitHub.
The rate limit for a given authenticated client is 5.000 calls per hour. This limit is **never** reached when looking for new release (thanks to the [conditional requests](https://developer.github.com/v3/#conditional-requests) of the GitHub API) on a daily basis.
But when new user sign in, we need to sync all its starred repositories and also all their releases / tags. And here come the gourmand part:
- one call for the list of release
- one call to retrieve information of each tag (if the repo doesn't have release)
- one call for each release to convert markdown text to html
Let's say the repo:
- has 50 tags: 1 (get tag list) + 50 (get commit information) + 50 (convert markdown) = 101 calls.
- has 50 releases: 1 (get tag list) + 50 (get each release) + 50 (convert markdown) = 101 calls.
And keep in mind that some repos got also 1.000+ tags (!!).
To avoid hitting the limit in such case and wait 1 hour to be able to make requests again I created the [GitHub Client Discovery class](src/AppBundle/Github/ClientDiscovery.php).
It aims to find the best client with enough rate limit remain (defined as 50).
- it first checks using the GitHub OAuth app
- then it checks using all user GitHub token
Which means, if you have 5 users on the app, you'll be able to make (1 + 5) x 5.000 = 30.000 calls per hour
================================================
FILE: bin/console
================================================
#!/usr/bin/env php
=8.2",
"cache/adapter-common": "dev-psr-cache-v3",
"cache/tag-interop": "dev-psr-cache-v3",
"doctrine/doctrine-bundle": "^2.0",
"doctrine/doctrine-migrations-bundle": "^3.0",
"doctrine/orm": "^3.5",
"knplabs/github-api": "^3.0",
"knplabs/knp-time-bundle": "^2.4",
"knpuniversity/oauth2-client-bundle": "^2.1",
"laminas/laminas-code": "^4.5",
"league/oauth2-github": "^3.0",
"marcw/rss-writer": "dev-fix/symfony-6",
"php-http/guzzle7-adapter": "^1.1",
"predis/predis": "^3.2",
"ramsey/uuid": "^4.0",
"sentry/sentry-symfony": "^5.0",
"snc/redis-bundle": "^4.10",
"symfony/amqp-messenger": "*",
"symfony/asset": "*",
"symfony/dotenv": "*",
"symfony/expression-language": "*",
"symfony/flex": "^2.5",
"symfony/monolog-bundle": "^4.0",
"symfony/polyfill-apcu": "^1.0",
"symfony/polyfill-php80": "^1.27",
"symfony/runtime": "*",
"symfony/security-bundle": "*",
"symfony/translation": "*",
"symfony/twig-bundle": "*",
"symfony/validator": "*",
"symfony/yaml": "*",
"twig/extra-bundle": "^2.12|^3.0",
"twig/twig": "^2.0|^3.0"
},
"require-dev": {
"doctrine/doctrine-fixtures-bundle": "^4.1",
"friendsofphp/php-cs-fixer": "~3.0",
"phpstan/extension-installer": "^1.0",
"phpstan/phpstan": "^2.0",
"phpstan/phpstan-deprecation-rules": "^2.0",
"phpstan/phpstan-doctrine": "^2.0",
"phpstan/phpstan-phpunit": "^2.0",
"phpstan/phpstan-symfony": "^2.0",
"phpunit/phpunit": "^11",
"rector/rector": "^2.1",
"symfony/browser-kit": "*",
"symfony/css-selector": "*",
"symfony/debug-bundle": "*",
"symfony/phpunit-bridge": "*",
"symfony/web-profiler-bundle": "*"
},
"conflict": {
"symfony/symfony": "*"
},
"scripts": {
"auto-scripts": {
"cache:clear": "symfony-cmd",
"assets:install %PUBLIC_DIR%": "symfony-cmd"
},
"post-install-cmd": [
"@auto-scripts"
],
"post-update-cmd": [
"@auto-scripts"
]
},
"config": {
"bin-dir": "bin",
"sort-packages": true,
"allow-plugins": {
"phpstan/extension-installer": true,
"symfony/flex": true,
"symfony/runtime": true,
"php-http/discovery": true
}
},
"repositories": [
{
"type": "vcs",
"url": "https://github.com/j0k3r/rss-writer"
},
{
"type": "vcs",
"url": "https://github.com/j0k3r/adapter-common"
},
{
"type": "vcs",
"url": "https://github.com/j0k3r/tag-interop"
}
],
"extra": {
"symfony": {
"allow-contrib": true,
"require": "7.4.*"
}
}
}
================================================
FILE: config/bundles.php
================================================
['all' => true],
DoctrineBundle::class => ['all' => true],
DoctrineMigrationsBundle::class => ['all' => true],
KnpTimeBundle::class => ['all' => true],
KnpUOAuth2ClientBundle::class => ['all' => true],
SentryBundle::class => ['prod' => true],
SncRedisBundle::class => ['all' => true],
MonologBundle::class => ['all' => true],
DoctrineFixturesBundle::class => ['dev' => true, 'test' => true],
TwigBundle::class => ['all' => true],
TwigExtraBundle::class => ['all' => true],
SecurityBundle::class => ['all' => true],
DebugBundle::class => ['dev' => true, 'test' => true],
WebProfilerBundle::class => ['dev' => true, 'test' => true],
];
================================================
FILE: config/packages/cache.yaml
================================================
framework:
cache:
# Unique name of your app: used to compute stable namespaces for cache keys.
#prefix_seed: your_vendor_name/app_name
# The "app" cache stores to the filesystem by default.
# The data in this cache should persist between deploys.
# Other options include:
# Redis
#app: cache.adapter.redis
#default_redis_provider: redis://localhost
# APCu (not recommended with heavy random-write workloads as memory fragmentation can cause perf issues)
#app: cache.adapter.apcu
# Namespaced pools use the above "app" backend by default
#pools:
#my.dedicated.cache: null
================================================
FILE: config/packages/debug.yaml
================================================
when@dev:
debug:
# Forwards VarDumper Data clones to a centralized server allowing to inspect dumps on CLI or in your browser.
# See the "server:dump" command to start a new server.
dump_destination: "tcp://%env(VAR_DUMPER_SERVER)%"
================================================
FILE: config/packages/doctrine.yaml
================================================
doctrine:
dbal:
url: '%env(resolve:DATABASE_URL)%'
# driver: pdo_mysql
# host: "%database_host%"
# port: "%database_port%"
# dbname: "%database_name%"
# user: "%database_user%"
# password: "%database_password%"
charset: utf8mb4
server_version: 5.7
default_table_options:
charset: utf8mb4
collate: utf8mb4_unicode_ci
# backtrace queries in profiler (increases memory usage per request)
profiling_collect_backtrace: '%kernel.debug%'
use_savepoints: true
orm:
auto_generate_proxy_classes: true
enable_lazy_ghost_objects: true
report_fields_where_declared: true
validate_xml_mapping: true
naming_strategy: doctrine.orm.naming_strategy.underscore_number_aware
identity_generation_preferences:
Doctrine\DBAL\Platforms\PostgreSQLPlatform: identity
auto_mapping: true
mappings:
App:
type: attribute
is_bundle: false
dir: '%kernel.project_dir%/src/Entity'
prefix: 'App\Entity'
alias: App
controller_resolver:
auto_mapping: false
when@test:
doctrine:
dbal:
logging: false
# "TEST_TOKEN" is typically set by ParaTest
# dbname_suffix: '_test%env(default::TEST_TOKEN)%'
when@prod:
doctrine:
orm:
auto_generate_proxy_classes: false
proxy_dir: '%kernel.build_dir%/doctrine/orm/Proxies'
metadata_cache_driver:
type: pool
pool: doctrine.system_cache_pool
query_cache_driver:
type: pool
pool: doctrine.system_cache_pool
result_cache_driver:
type: pool
pool: doctrine.result_cache_pool
framework:
cache:
pools:
doctrine.result_cache_pool:
adapter: cache.app
doctrine.system_cache_pool:
adapter: cache.system
================================================
FILE: config/packages/doctrine_migrations.yaml
================================================
doctrine_migrations:
migrations_paths:
# namespace is arbitrary but should be different from App\Migrations
# as migrations classes should NOT be autoloaded
'DoctrineMigrations': '%kernel.project_dir%/migrations'
enable_profiler: false
================================================
FILE: config/packages/framework.yaml
================================================
# see https://symfony.com/doc/current/reference/configuration/framework.html
framework:
secret: '%env(APP_SECRET)%'
annotations: false
http_method_override: true
handle_all_throwables: true
# Enables session support. Note that the session will ONLY be started if you read or write from it.
# Remove or comment this section to explicitly disable session support.
session:
name: banditore
storage_factory_id: session.storage.factory.native
save_path: "%kernel.project_dir%/var/sessions/%kernel.environment%"
cookie_secure: auto
cookie_samesite: lax
#esi: true
#fragments: true
when@test:
framework:
test: true
session:
storage_factory_id: session.storage.factory.mock_file
================================================
FILE: config/packages/github_api.yaml
================================================
services:
Github\Client:
arguments:
- '@Github\HttpClient\Builder'
# Uncomment to enable authentication
#calls:
# - ['authenticate', ['%env(GITHUB_USERNAME)%', '%env(GITHUB_SECRET)%', '%env(GITHUB_AUTH_METHOD)%']]
Github\HttpClient\Builder:
arguments:
- '@?Http\Client\HttpClient'
- '@?Http\Message\RequestFactory'
- '@?Http\Message\StreamFactory'
================================================
FILE: config/packages/http_discovery.yaml
================================================
services:
Psr\Http\Message\RequestFactoryInterface: '@http_discovery.psr17_factory'
Psr\Http\Message\ResponseFactoryInterface: '@http_discovery.psr17_factory'
Psr\Http\Message\ServerRequestFactoryInterface: '@http_discovery.psr17_factory'
Psr\Http\Message\StreamFactoryInterface: '@http_discovery.psr17_factory'
Psr\Http\Message\UploadedFileFactoryInterface: '@http_discovery.psr17_factory'
Psr\Http\Message\UriFactoryInterface: '@http_discovery.psr17_factory'
http_discovery.psr17_factory:
class: Http\Discovery\Psr17Factory
================================================
FILE: config/packages/knpu_oauth2_client.yaml
================================================
knpu_oauth2_client:
clients:
# will create service: "knpu.oauth2.client.github"
# an instance of: KnpU\OAuth2ClientBundle\Client\Provider\GithubClient
github:
type: github
client_id: "%env(GITHUB_CLIENT_ID)%"
client_secret: "%env(GITHUB_CLIENT_SECRET)%"
# a route name you'll create
redirect_route: github_callback
redirect_params: {}
================================================
FILE: config/packages/messenger.yaml
================================================
framework:
messenger:
# Uncomment this (and the failed transport below) to send failed messages to this transport for later handling.
# failure_transport: failed
transports:
sync_starred_repos:
dsn: '%env(MESSENGER_TRANSPORT_DSN)%/sync_starred_repos'
options:
exchange:
name: banditore.sync_starred_repos
type: direct
queues:
banditore.sync_starred_repos: ~
sync_versions:
dsn: '%env(MESSENGER_TRANSPORT_DSN)%/sync_versions'
options:
exchange:
name: banditore.sync_versions
type: direct
queues:
banditore.sync_versions: ~
# https://symfony.com/doc/current/messenger.html#transport-configuration
# async: '%env(MESSENGER_TRANSPORT_DSN)%'
# failed: 'doctrine://default?queue_name=failed'
# sync: 'sync://'
routing:
'App\Message\StarredReposSync': sync_starred_repos
'App\Message\VersionsSync': sync_versions
buses:
command_bus:
middleware:
- doctrine_ping_connection
- doctrine_close_connection
when@test:
framework:
messenger:
transports:
sync_starred_repos: 'in-memory://'
sync_versions: 'in-memory://'
================================================
FILE: config/packages/monolog.yaml
================================================
monolog:
channels:
- deprecation # Deprecations are logged in the dedicated "deprecation" channel when it exists
when@dev:
monolog:
handlers:
main:
type: stream
path: "%kernel.logs_dir%/%kernel.environment%.log"
level: debug
channels: ["!event"]
console:
type: console
process_psr_3_messages: false
channels: ["!event", "!doctrine", "!console"]
when@test:
monolog:
handlers:
main:
type: fingers_crossed
action_level: error
handler: nested
excluded_http_codes: [404, 405]
channels: ["!event"]
nested:
type: stream
path: "%kernel.logs_dir%/%kernel.environment%.log"
level: debug
when@prod:
monolog:
handlers:
main:
type: fingers_crossed
action_level: error
handler: nested
excluded_http_codes: [404, 405]
channels: ["!deprecation"]
buffer_size: 50 # How many messages should be saved? Prevent memory leaks
nested:
type: rotating_file
path: "%kernel.logs_dir%/%kernel.environment%.log"
level: debug
max_files: 10
console:
type: console
process_psr_3_messages: false
channels: ["!event", "!doctrine"]
deprecation:
type: stream
channels: [deprecation]
path: "%kernel.logs_dir%/deprecation.log"
================================================
FILE: config/packages/routing.yaml
================================================
framework:
router:
# Configure how to generate URLs in non-HTTP contexts, such as CLI commands.
# See https://symfony.com/doc/current/routing.html#generating-urls-in-commands
default_uri: '%env(DEFAULT_URI)%'
when@prod:
framework:
router:
strict_requirements: null
================================================
FILE: config/packages/security.yaml
================================================
security:
# https://symfony.com/doc/current/security.html#registering-the-user-hashing-passwords
password_hashers:
Symfony\Component\Security\Core\User\PasswordAuthenticatedUserInterface: 'auto'
# https://symfony.com/doc/current/security.html#loading-the-user-the-user-provider
providers:
github_provider:
entity:
class: App\Entity\User
property: githubId
firewalls:
dev:
# Ensure dev tools and static assets are always allowed
pattern: ^/(_profiler|_wdt|assets|build|css|images|js)/
security: false
main:
lazy: true
custom_authenticators:
- App\Security\GithubAuthenticator
logout:
path: logout
# Activate different ways to authenticate:
# https://symfony.com/doc/current/security.html#the-firewall
# https://symfony.com/doc/current/security/impersonating_user.html
# switch_user: true
# Note: Only the *first* matching rule is applied
access_control:
# - { path: ^/admin, roles: ROLE_ADMIN }
# - { path: ^/profile, roles: ROLE_USER }
when@test:
security:
password_hashers:
# Password hashers are resource-intensive by design to ensure security.
# In tests, it's safe to reduce their cost to improve performance.
Symfony\Component\Security\Core\User\PasswordAuthenticatedUserInterface:
algorithm: auto
cost: 4 # Lowest possible value for bcrypt
time_cost: 3 # Lowest possible value for argon
memory_cost: 10 # Lowest possible value for argon
================================================
FILE: config/packages/sentry.yaml
================================================
when@prod:
sentry:
dsn: '%env(SENTRY_DSN)%'
options:
integrations:
- 'Sentry\Integration\IgnoreErrorsIntegration'
- 'Symfony\Component\ErrorHandler\Error\FatalError'
- 'Symfony\Component\Debug\Exception\FatalErrorException'
# If you are using Monolog, you also need these additional configuration and services to log the errors correctly:
# https://docs.sentry.io/platforms/php/guides/symfony/#monolog-integration
register_error_listener: false
register_error_handler: false
# this hooks into critical paths of the framework (and vendors) to perform
# automatic instrumentation (there might be some performance penalty)
# https://docs.sentry.io/platforms/php/guides/symfony/performance/instrumentation/automatic-instrumentation/
tracing:
enabled: false
monolog:
handlers:
sentry:
type: service
id: Sentry\Monolog\Handler
level: !php/const Monolog\Logger::ERROR
services:
Sentry\Integration\IgnoreErrorsIntegration:
arguments:
$options:
ignore_exceptions:
- Symfony\Component\HttpKernel\Exception\NotFoundHttpException
Sentry\Monolog\Handler:
arguments:
$hub: '@Sentry\State\HubInterface'
$level: !php/const Monolog\Logger::ERROR
$bubble: false
================================================
FILE: config/packages/snc_redis.yaml
================================================
# Define your clients here. The example below connects to database 0 of the default Redis server.
#
# See https://github.com/snc/SncRedisBundle/blob/master/docs/README.md for instructions on
# how to configure the bundle.
snc_redis:
clients:
guzzle_cache:
type: predis
alias: guzzle_cache
dsn: "%env(REDIS_URL_GUZZLE_CACHE)%"
app_cache:
type: predis
alias: app_cache
dsn: "%env(REDIS_URL_APP_CACHE)%"
================================================
FILE: config/packages/twig.yaml
================================================
twig:
file_name_pattern: '*.twig'
when@test:
twig:
strict_variables: true
================================================
FILE: config/packages/validator.yaml
================================================
framework:
validation:
enable_attributes: true
email_validation_mode: html5
# Enables validator auto-mapping support.
# For instance, basic validation constraints will be inferred from Doctrine's metadata.
auto_mapping:
App\Entity\: []
when@test:
framework:
validation:
not_compromised_password: false
================================================
FILE: config/packages/web_profiler.yaml
================================================
when@dev:
web_profiler:
toolbar: true
framework:
profiler:
collect_serializer_data: true
when@test:
web_profiler:
toolbar: false
intercept_redirects: false
framework:
profiler:
enabled: false
collect: false
collect_serializer_data: true
================================================
FILE: config/preload.php
================================================
[
* 'App\\' => [
* 'resource' => '../src/',
* ],
* ],
* ]);
* ```
*
* @psalm-type ImportsConfig = list
* $paginator = new Paginator(array(
* 'itemTotalCallback' => function () {
* // ...
* },
* 'sliceCallback' => function ($offset, $length) {
* // ...
* },
* 'itemsPerPage' => 10,
* 'pagesInRange' => 5
* ));
*
*/
public function __construct(?array $config = null)
{
if (\array_key_exists('itemTotalCallback', $config)) {
$this->setItemTotalCallback($config['itemTotalCallback']);
}
if (\array_key_exists('sliceCallback', $config)) {
$this->setSliceCallback($config['sliceCallback']);
}
if (\array_key_exists('itemsPerPage', $config)) {
$this->setItemsPerPage($config['itemsPerPage']);
}
if (\array_key_exists('pagesInRange', $config)) {
$this->setPagesInRange($config['pagesInRange']);
}
}
public function paginate($currentPageNumber = 1)
{
if (!$this->itemTotalCallback instanceof \Closure) {
throw new CallbackNotFoundException('Item total callback not found, set it using Paginator::setItemTotalCallback()');
}
if (!$this->sliceCallback instanceof \Closure) {
throw new CallbackNotFoundException('Slice callback not found, set it using Paginator::setSliceCallback()');
}
if (!\is_int($currentPageNumber)) {
throw new \InvalidArgumentException(\sprintf('Current page number must be of type integer, %s given', \gettype($currentPageNumber)));
}
if ($currentPageNumber <= 0) {
throw new InvalidPageNumberException(\sprintf('Current page number must have a value of 1 or more, %s given', $currentPageNumber));
}
$beforeQueryCallback = $this->beforeQueryCallback instanceof \Closure
? $this->beforeQueryCallback
: static function (): void {}
;
$afterQueryCallback = $this->afterQueryCallback instanceof \Closure
? $this->afterQueryCallback
: static function (): void {}
;
$pagination = new Pagination();
$itemTotalCallback = $this->itemTotalCallback;
$beforeQueryCallback($this, $pagination);
$totalNumberOfItems = (int) $itemTotalCallback($pagination);
$afterQueryCallback($this, $pagination);
$numberOfPages = (int) ceil($totalNumberOfItems / $this->itemsPerPage);
$pagesInRange = $this->pagesInRange;
if ($pagesInRange > $numberOfPages) {
$pagesInRange = $numberOfPages;
}
$change = (int) ceil($pagesInRange / 2);
if (($currentPageNumber - $change) > ($numberOfPages - $pagesInRange)) {
$pages = range(($numberOfPages - $pagesInRange) + 1, $numberOfPages);
} else {
if (($currentPageNumber - $change) < 0) {
$change = $currentPageNumber;
}
$offset = $currentPageNumber - $change;
$pages = range($offset + 1, $offset + $pagesInRange);
}
$offset = ($currentPageNumber - 1) * $this->itemsPerPage;
$sliceCallback = $this->sliceCallback;
$beforeQueryCallback($this, $pagination);
if (-1 === $this->itemsPerPage) {
$items = $sliceCallback(0, 999999999, $pagination);
} else {
$items = $sliceCallback($offset, $this->itemsPerPage, $pagination);
}
if ($items instanceof \Iterator) {
$items = iterator_to_array($items);
}
$afterQueryCallback($this, $pagination);
$pagination
->setItems($items)
->setPages($pages)
->setTotalNumberOfPages($numberOfPages)
->setCurrentPageNumber($currentPageNumber)
->setFirstPageNumber(1)
->setLastPageNumber($numberOfPages)
->setItemsPerPage($this->itemsPerPage)
->setTotalNumberOfItems($totalNumberOfItems)
->setFirstPageNumberInRange(min($pages))
->setLastPageNumberInRange(max($pages))
;
$previousPageNumber = null;
if (($currentPageNumber - 1) > 0) {
$pagination->setPreviousPageNumber($currentPageNumber - 1);
}
$nextPageNumber = null;
if (($currentPageNumber + 1) <= $numberOfPages) {
$pagination->setNextPageNumber($currentPageNumber + 1);
}
return $pagination;
}
public function getSliceCallback()
{
return $this->sliceCallback;
}
public function setSliceCallback(\Closure $sliceCallback)
{
$this->sliceCallback = $sliceCallback;
return $this;
}
public function getItemTotalCallback()
{
return $this->itemTotalCallback;
}
public function getBeforeQueryCallback()
{
return $this->beforeQueryCallback;
}
public function setBeforeQueryCallback(\Closure $beforeQueryCallback)
{
$this->beforeQueryCallback = $beforeQueryCallback;
return $this;
}
public function getAfterQueryCallback()
{
return $this->afterQueryCallback;
}
public function setAfterQueryCallback(\Closure $afterQueryCallback)
{
$this->afterQueryCallback = $afterQueryCallback;
return $this;
}
public function setItemTotalCallback(\Closure $itemTotalCallback)
{
$this->itemTotalCallback = $itemTotalCallback;
return $this;
}
public function getItemsPerPage()
{
return $this->itemsPerPage;
}
public function setItemsPerPage($itemsPerPage)
{
if (!\is_int($itemsPerPage)) {
throw new \InvalidArgumentException(\sprintf('Items per page must be of type integer, %s given', \gettype($itemsPerPage)));
}
$this->itemsPerPage = $itemsPerPage;
return $this;
}
public function getPagesInRange()
{
return $this->pagesInRange;
}
public function setPagesInRange($pagesInRange)
{
if (!\is_int($pagesInRange)) {
throw new \InvalidArgumentException(\sprintf('Pages in range must be of type integer, %s given', \gettype($pagesInRange)));
}
$this->pagesInRange = $pagesInRange;
return $this;
}
}
================================================
FILE: src/Pagination/PaginatorInterface.php
================================================
*/
interface PaginatorInterface
{
/**
* Run paginate algorithm using the current page number.
*
* @param int $currentPageNumber Page number, usually passed from the current request
*
* @throws \InvalidArgumentException
* @throws InvalidPageNumberException
*
* @return Pagination Collection of items returned by the slice callback with pagination meta information
*/
public function paginate($currentPageNumber = 1);
/**
* Get sliceCallback.
*
* @return \Closure
*/
public function getSliceCallback();
/**
* Set sliceCallback.
*
* @return $this
*/
public function setSliceCallback(\Closure $sliceCallback);
/**
* Get itemTotalCallback.
*
* @return \Closure
*/
public function getItemTotalCallback();
/**
* Set itemTotalCallback.
*
* @return $this
*/
public function setItemTotalCallback(\Closure $itemTotalCallback);
/**
* @return \Closure
*/
public function getBeforeQueryCallback();
/**
* @return $this
*/
public function setBeforeQueryCallback(\Closure $beforeQueryCallback);
/**
* @return \Closure
*/
public function getAfterQueryCallback();
/**
* @return $this
*/
public function setAfterQueryCallback(\Closure $afterQueryCallback);
/**
* Get itemsPerPage.
*
* @return int
*/
public function getItemsPerPage();
/**
* Set itemsPerPage.
*
* @param int $itemsPerPage
*
* @throws \InvalidArgumentException
*
* @return $this
*/
public function setItemsPerPage($itemsPerPage);
/**
* Get pagesInRange.
*
* @return int
*/
public function getPagesInRange();
/**
* Set pagesInRange.
*
* @param int $pagesInRange
*
* @throws \InvalidArgumentException
*
* @return $this
*/
public function setPagesInRange($pagesInRange);
}
================================================
FILE: src/PubSubHubbub/Publisher.php
================================================
router->getContext();
$context->setHost($host);
$context->setScheme($scheme);
}
/**
* Ping available hub when new items are cached.
*
* http://nathangrigg.net/2012/09/real-time-publishing/
*
* @param array $repoIds Id of repo from the database
*
* @return bool
*/
public function pingHub(array $repoIds)
{
if (empty($this->hub) || empty($repoIds)) {
return false;
}
$urls = $this->retrieveFeedUrls($repoIds);
// ping publisher
// https://github.com/pubsubhubbub/php-publisher/blob/master/library/Publisher.php
$params = 'hub.mode=publish';
foreach ($urls as $url) {
$params .= '&hub.url=' . $url;
}
$response = $this->client->post(
$this->hub,
[
'http_errors' => false,
'body' => $params,
'headers' => [
'Content-Type' => 'application/x-www-form-urlencoded',
'User-Agent' => 'Banditore/1.0',
],
]
);
// hub should response 204 if everything went fine
return !(204 !== $response->getStatusCode());
}
/**
* Retrieve user feed urls from a list of repository ids.
*
* @return array
*/
private function retrieveFeedUrls(array $repoIds)
{
$users = $this->userRepository->findByRepoIds($repoIds);
$urls = [];
foreach ($users as $user) {
$urls[] = $this->router->generate(
'rss_user',
['uuid' => $user['uuid']],
UrlGeneratorInterface::ABSOLUTE_URL
);
}
return $urls;
}
}
================================================
FILE: src/Repository/RepoRepository.php
================================================
*/
class RepoRepository extends ServiceEntityRepository
{
public function __construct(ManagerRegistry $registry)
{
parent::__construct($registry, Repo::class);
}
/**
* Retrieve all repositories to be fetched for new release.
*
* @return array
*/
public function findAllForRelease()
{
$data = $this->createQueryBuilder('r')
->select('r.id')
->where('r.removedAt IS NULL')
->getQuery()
->getArrayResult();
$return = [];
foreach ($data as $oneData) {
$return[] = $oneData['id'];
}
return $return;
}
/**
* Count total repos.
*
* @return int
*/
public function countTotal()
{
return (int) $this->createQueryBuilder('r')
->select('COUNT(r.id) as total')
->getQuery()
->getSingleScalarResult();
}
/**
* Retrieve repos with the most releases.
* Used for stats.
*
* @return array
*/
public function mostVersionsPerRepo()
{
return $this->createQueryBuilder('r')
->select('r.fullName', 'r.description', 'r.ownerAvatar')
->addSelect('(SELECT COUNT(v.id)
FROM App\Entity\Version v
WHERE v.repo = r.id) AS total'
)
->groupBy('r.fullName', 'r.description', 'r.ownerAvatar', 'total')
->orderBy('total', 'desc')
->setMaxResults(5)
->getQuery()
->getArrayResult();
}
}
================================================
FILE: src/Repository/StarRepository.php
================================================
*/
class StarRepository extends ServiceEntityRepository
{
public function __construct(ManagerRegistry $registry)
{
parent::__construct($registry, Star::class);
}
/**
* Retrieve all repos starred by a user.
*
* @param int $userId User id
*
* @return array
*/
public function findAllByUser($userId)
{
$repos = $this->createQueryBuilder('s')
->select('r.id')
->leftJoin('s.repo', 'r')
->where('s.user = ' . $userId)
->getQuery()
->getArrayResult();
$res = [];
foreach ($repos as $repo) {
$res[] = $repo['id'];
}
return $res;
}
/**
* Remove stars for a user.
*/
public function removeFromUser(array $repoIds, int $userId): void
{
$this->createQueryBuilder('s')
->delete()
->where('s.repo IN (:ids)')->setParameter('ids', $repoIds)
->andWhere('s.user = :userId')->setParameter('userId', $userId)
->getQuery()
->execute();
}
/**
* Count total stars.
*
* @return int
*/
public function countTotal()
{
return (int) $this->createQueryBuilder('s')
->select('COUNT(s.id) as total')
->getQuery()
->getSingleScalarResult();
}
public function findOneByUserAndRepo(int $userId, int $repoId): ?Star
{
$stars = $this->createQueryBuilder('s')
->leftJoin('s.repo', 'r')
->where('s.user = :userId')->setParameter('userId', $userId)
->andWhere('r.id = :repoId')->setParameter('repoId', $repoId)
->setMaxResults(1)
->getQuery()
->getResult();
return $stars[0] ?? null;
}
}
================================================
FILE: src/Repository/UserRepository.php
================================================
*/
class UserRepository extends ServiceEntityRepository
{
public function __construct(ManagerRegistry $registry)
{
parent::__construct($registry, User::class);
}
/**
* Retrieve user.
*
* @return array
*/
public function findByRepoIds(array $repoIds)
{
return $this->createQueryBuilder('u')
->select('DISTINCT u.uuid')
->leftJoin('u.stars', 's')
->where('s.repo IN (:ids)')->setParameter('ids', $repoIds)
->andWhere('s.ignoredInFeed = :ignoredInFeed')->setParameter('ignoredInFeed', false)
->getQuery()
->getArrayResult();
}
/**
* Retrieve all users to be synced.
* We only retrieve ids to be as fast as possible.
*
* @return array
*/
public function findAllToSync()
{
$data = $this->createQueryBuilder('u')
->select('u.id')
->where('u.removedAt IS NULL')
->getQuery()
->getArrayResult();
$return = [];
foreach ($data as $oneData) {
$return[] = $oneData['id'];
}
return $return;
}
/**
* Retrieve all tokens available.
* This is used for the GithubClientDiscovery.
*
* @return array
*/
public function findAllTokens()
{
return $this->createQueryBuilder('u')
->select('u.id', 'u.username', 'u.accessToken')
->where('u.removedAt IS NULL')
->getQuery()
->enableResultCache()
->setResultCacheLifetime(10 * 60)
->getArrayResult();
}
/**
* Count total users.
*
* @return int
*/
public function countTotal()
{
return (int) $this->createQueryBuilder('u')
->select('COUNT(u.id) as total')
->getQuery()
->getSingleScalarResult();
}
}
================================================
FILE: src/Repository/VersionRepository.php
================================================
*/
class VersionRepository extends ServiceEntityRepository
{
public function __construct(ManagerRegistry $registry)
{
parent::__construct($registry, Version::class);
}
/**
* Find one version for a given tag name and repo id.
* This is exactly the same as `findOneBy` but this one use a result cache.
* Version doesn't change after being inserted and since we check to many times for a version
* it's faster to store result in a cache.
*
* @param string $tagName Tag name to search, like v1.0.0
* @param int $repoId Repository ID
*
* @return int|null
*/
public function findExistingOne($tagName, $repoId)
{
$query = $this->createQueryBuilder('v')
->select('v.id')
->where('v.repo = :repoId')->setParameter('repoId', $repoId)
->andWhere('v.tagName = :tagName')->setParameter('tagName', $tagName)
->setMaxResults(1)
->getQuery()
;
return $query->getOneOrNullResult(AbstractQuery::HYDRATE_SINGLE_SCALAR);
}
/**
* Find all versions available for the given user.
*
* @param int $userId
* @param int $offset
* @param int $length
*
* @return array
*/
public function findForUser($userId, $offset = 0, $length = 20)
{
return $this->createQueryBuilder('v')
->select('r.id AS repoId', 'v.tagName', 'v.name', 'v.createdAt', 'v.body', 'v.prerelease', 's.ignoredInFeed', 'r.fullName', 'r.ownerAvatar', 'r.ownerAvatar', 'r.homepage', 'r.language', 'r.description')
->leftJoin('v.repo', 'r')
->leftJoin('r.stars', 's')
->where('s.user = :userId')->setParameter('userId', $userId)
->orderBy('v.createdAt', 'desc')
->setFirstResult($offset)
->setMaxResults($length)
->getQuery()
->getArrayResult();
}
/**
* Count all versions available for the given user.
* Used in the dashboard pagination and auth process.
*
* @param int $userId
*
* @return int
*/
public function countForUser($userId)
{
return (int) $this->createQueryBuilder('v')
->select('COUNT(v.id)')
->leftJoin('v.repo', 'r')
->leftJoin('r.stars', 's')
->where('s.user = :userId')->setParameter('userId', $userId)
->getQuery()
->getSingleScalarResult();
}
/**
* Find all versions available in the feed for the given user.
*/
public function findForFeedUser(int $userId, int $offset = 0, int $length = 20): array
{
return $this->createQueryBuilder('v')
->select('r.id AS repoId', 'v.tagName', 'v.name', 'v.createdAt', 'v.body', 'v.prerelease', 'r.fullName', 'r.ownerAvatar', 'r.ownerAvatar', 'r.homepage', 'r.language', 'r.description')
->leftJoin('v.repo', 'r')
->leftJoin('r.stars', 's')
->where('s.user = :userId')->setParameter('userId', $userId)
->andWhere('s.ignoredInFeed = :ignoredInFeed')->setParameter('ignoredInFeed', false)
->orderBy('v.createdAt', 'desc')
->setFirstResult($offset)
->setMaxResults($length)
->getQuery()
->getArrayResult();
}
/**
* Retrieve latest version of each repo.
*
* @param int $length Number of items
*
* @return array
*/
public function findLastVersionForEachRepo($length = 10)
{
$query = '
SELECT v1.tagName, v1.name, v1.createdAt, r.fullName, r.description, r.ownerAvatar, v1.prerelease
FROM App\Entity\Version v1
LEFT JOIN App\Entity\Version v2 WITH (v1.repo = v2.repo AND v1.createdAt < v2.createdAt)
LEFT JOIN App\Entity\Repo r WITH r.id = v1.repo
WHERE v2.repo IS NULL
ORDER BY v1.createdAt DESC
';
return $this->getEntityManager()->createQuery($query)
->setFirstResult(0)
->setMaxResults($length)
->getArrayResult();
}
/**
* Count total versions.
*
* @return int
*/
public function countTotal()
{
return (int) $this->createQueryBuilder('v')
->select('COUNT(v.id) as total')
->getQuery()
->getSingleScalarResult();
}
/**
* Retrieve the latest version saved.
*
* @return array|null
*/
public function findLatest()
{
return $this->createQueryBuilder('v')
->select('v.createdAt')
->orderBy('v.createdAt', 'desc')
->setMaxResults(1)
->getQuery()
->getOneOrNullResult();
}
}
================================================
FILE: src/Rss/Generator.php
================================================
addExtension(
(new AtomLink())
->setRel('self')
->setHref($feedUrl)
->setType('application/rss+xml')
);
$channel->addExtension(
(new AtomLink())
->setRel('hub')
->setHref('http://pubsubhubbub.appspot.com/')
);
$channel->addExtension(
(new Webfeeds())
->setLogo($user->getAvatar())
->setIcon($user->getAvatar())
->setAccentColor('10556B')
);
$channel->setTitle(str_replace('%USERNAME%', $user->getUsername(), self::CHANNEL_TITLE))
->setLink($feedUrl)
->setDescription(str_replace('%USERNAME%', $user->getUsername(), self::CHANNEL_DESCRIPTION))
->setLanguage('en')
->setCopyright('(c) ' . (new \DateTime())->format('Y') . ' banditore')
->setLastBuildDate(isset($releases[0]) ? $releases[0]['createdAt'] : new \DateTime())
->setGenerator('banditore');
foreach ($releases as $release) {
// build repo top information
$repoHome = $release['homepage'] ? '(' . $release['homepage'] . ')' : '';
$repoLanguage = $release['language'] ? '#' . $release['language'] . '
' : ''; $repoInformation = '|
|
' . $release['fullName'] . '
' . $repoHome . ' ' . $release['description'] . ' ' . $repoLanguage . ' |
Looks like we have a problem here.

You might want to return to the homepage.
And if you are already on the homepage, it might be a bigger problem. So don't move.
{{ pagination_render( pagination, 'dashboard', 'page', app.request.query.all ) }}
{% endif %} {% else %}Here are some sample display.
{% endif %}| Repository | Last version | Published at | Included in feed |
|---|---|---|---|
|
|
First release (v1.0.0) | ||
|
|
Prepare first release (v1.0.0-alpha.1) pre-release |
{{ pagination_render( pagination, 'dashboard', 'page', app.request.query.all ) }}
{% endif %}Banditore will check for new release every 10 minutes.
Banditore will sync your starred repos every 5 minutes. You can also force it by logged out / logged in.
First, they can still be in sync. This could be the case if it's the very first time you logged in here. Second, not all repos got tag or release (which is sad). In that case, they'll never show up on your dashboard.
Feel free to open an issue.
We gather new releases from your starred GitHub repositories and generate an Atom feed with them.
Just for you.
When a new release / tag is available, you'll know it right away. So you won't forget to update your project.
I know you're tired of emails, popups & browser / mobile notifications. RSS feed is soft notification that won't bother you.
We all do that: you star a repo and the next few days you already forget about it. That's over. Any new release will remind you about it!
Banditore is full open source. If you don't want to use this website, you can install it on your own server.
When you first login, we retrieve minimal information from you (name, username & avatar). Then we fetch your stars & their associated repository.
At least twice per hour, we retrieve new release or tag for your repository using your token. Once we have gathered all these information, we build a feed with them, ordered by published date of each release.
Some release contains markdown information which will be converted in HTML for a better rendering. For tag (which doesn't come with a body), we'll only display the tag name.
As I said, Banditore is open source. If something bother you or if you want to improve it, I'll be much happy to check your issue or review your PR!
Banditore is an italian word. It means a town crier (or in french un crieur public).
Wikipedia says: "A town crier, or bellman, is an officer of the court who makes public pronouncements as required by the court ".
We are not in court here but I think you get the idea. Banditore will makes "announcements" about new releases from repositories you starred. On every new release (or tag, because some people don't use the Release feature) it'll push a new item right away in your Atom feed.
| Number of repos | {{ counters.nbRepos }} |
| Numbers of releases | {{ counters.nbReleases }} |
| Average release per repo | {{ counters.avgReleasePerRepo }} |
| Average star per user | {{ counters.avgStarPerUser }} |
|
|
{{ repo.total }} |
Soon …
| Repository | Last version | Published at |
|---|
Use the correct package type for composer.
'), // TAG 1.0.2 // repos/release with tag 1.0.2 (which is not a release) new Response(404, ['Content-Type' => 'application/json'], (string) json_encode([ 'message' => 'Not Found', 'documentation_url' => 'https://developer.github.com/v3', ])), // retrieve tag information from the tag (since the release does not exist) $this->getOKResponse([ 'sha' => '694b8cc3983f52209029605300910507bec700b4', 'url' => 'https://api.github.com/repos/snc/SncRedisBundle/git/tags/694b8cc3983f52209029605300910507bec700b4', 'tagger' => [ 'name' => 'Erwin Mombay', 'email' => 'erwinm@google.com', 'date' => '2012-10-18T17:23:37Z', ], 'object' => [ 'sha' => '694b8cc3983f52209029605300910507bec700b5', 'type' => 'commit', 'url' => 'https://api.github.com/repos/snc/SncRedisBundle/git/commits/694b8cc3983f52209029605300910507bec700b5', ], 'tag' => '1.0.2', 'message' => "weekly release\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v2\n\niF4EABEIAAYFAliw58IACgkQ64qmmlZsB5VNFwD+L1M86cO76oohqSy4TCbubPAL\n6341glOKJpfkwyjQnUkBAPCTZSBbe8CFHLxLUvypIiQSMn+AIkPfvzvSEahA40Vz\n=SaF+\n-----END PGP SIGNATURE-----\n", ]), // markdown new Response(200, ['Content-Type' => 'text/html'], 'weekly release
'), // TAG 2.0.1 // now tag 2.0.1 which is a release $this->getOKResponse([ 'tag_name' => '2.0.1', 'name' => 'Trade-off memory for compute, Windows support, 24 distributions with cdf, variance etc., dtypes, zero-dimensional Tensors, Tensor-Variable merge, , faster distributed, perf and bug fixes, CuDNN 7.1', 'prerelease' => false, 'published_at' => '2017-02-19T13:27:32Z', 'body' => 'yay', ]), // markdown new Response(200, ['Content-Type' => 'text/html'], 'yay
'), // rate_limit $this->getOKResponse(['resources' => ['core' => ['reset' => time() + 1000, 'limit' => 200, 'remaining' => 10]]]), ]); } public function testProcessSuccessfulMessage(): void { $uow = $this->getMockBuilder(UnitOfWork::class) ->disableOriginalConstructor() ->getMock(); $uow->expects($this->exactly(3)) ->method('getScheduledEntityInsertions') ->willReturn([]); $em = $this->getMockBuilder(EntityManager::class) ->disableOriginalConstructor() ->getMock(); $em->expects($this->once()) ->method('isOpen') ->willReturn(false); // simulate a closing manager $em->expects($this->exactly(3)) ->method('getUnitOfWork') ->willReturn($uow); $doctrine = $this->getMockBuilder(Registry::class) ->disableOriginalConstructor() ->getMock(); $doctrine->expects($this->once()) ->method('getManager') ->willReturn($em); $doctrine->expects($this->once()) ->method('resetManager') ->willReturn($em); $repo = new Repo(); $repo->setId(123); $repo->setFullName('bob/wow'); $repo->setName('wow'); $repoRepository = $this->getMockBuilder(RepoRepository::class) ->disableOriginalConstructor() ->getMock(); $repoRepository->expects($this->once()) ->method('find') ->with(123) ->willReturn($repo); $versionRepository = $this->getMockBuilder(VersionRepository::class) ->disableOriginalConstructor() ->getMock(); $versionRepository->expects($this->exactly(4)) ->method('findExistingOne') ->willReturnCallback(static function ($tagName, $repoId) use ($repo) { // first version will exist, next one won't if ('1.0.0' === $tagName) { return new Version($repo); } }); $pubsubhubbub = $this->getMockBuilder(Publisher::class) ->disableOriginalConstructor() ->getMock(); $pubsubhubbub->expects($this->once()) ->method('pingHub') ->with([123]); $clientHandler = HandlerStack::create($this->getWorkingResponses()); $guzzleClient = new Client([ 'handler' => $clientHandler, ]); $httpClient = new Guzzle7Client($guzzleClient); $httpBuilder = new Builder($httpClient); $githubClient = new GithubClient($httpBuilder); $logger = new Logger('foo'); $logHandler = new TestHandler(); $logger->pushHandler($logHandler); $handler = new VersionsSyncHandler( $doctrine, $repoRepository, $versionRepository, $pubsubhubbub, $githubClient, $logger ); $handler->__invoke(new VersionsSync(123)); $records = $logHandler->getRecords(); $this->assertSame('Consume banditore.sync_versions message', $records[0]['message']); $this->assertSame('[10] Checkweekly release
', $versions[2]->getBody(), 'Version 1.0.2 does NOT have a PGP signature'); $this->assertSame('2.0.1', $versions[3]->getTagName(), 'Repo 666 has 4 version. Fourth one is 2.0.1'); } public function testFunctionalConsumerWithTagCaseInsensitive(): void { $this->restoreFunctionalState(); $responses = new MockHandler([ // rate_limit $this->getOKResponse(['resources' => ['core' => ['reset' => time() + 1000, 'limit' => 200, 'remaining' => 10]]]), // repo/tags $this->getOKResponse([[ 'name' => 'v2.11.0', 'zipball_url' => 'https://api.github.com/repos/mozilla/metrics-graphics/zipball/v2.11.0', 'tarball_url' => 'https://api.github.com/repos/mozilla/metrics-graphics/tarball/v2.11.0', ]]), // git/refs/tags $this->getOKResponse([ [ 'ref' => 'refs/tags/V1.1.0', 'url' => 'https://api.github.com/repos/mozilla/metrics-graphics/git/refs/tags/V1.1.0', 'object' => [ 'sha' => '6402716c3165eb90cdace5729a18706ea2921187', 'type' => 'commit', 'url' => 'https://api.github.com/repos/mozilla/metrics-graphics/git/commits/6402716c3165eb90cdace5729a18706ea2921187', ], ], [ 'ref' => 'refs/tags/v1.1.0', 'url' => 'https://api.github.com/repos/mozilla/metrics-graphics/git/refs/tags/v1.1.0', 'object' => [ 'sha' => '15a4703db568342043f156b5635d10b17ebe98cb', 'type' => 'commit', 'url' => 'https://api.github.com/repos/mozilla/metrics-graphics/git/commits/15a4703db568342043f156b5635d10b17ebe98cb', ], ], ]), // TAG V1.1.0 // now tag V1.1.0 which is a release $this->getOKResponse([ 'tag_name' => 'V1.1.0', 'name' => 'V1.1.0', 'prerelease' => false, 'published_at' => '2014-12-01T18:28:39Z', 'body' => 'This is the first release after our major push.', ]), // markdown new Response(200, ['Content-Type' => 'text/html'], 'This is the first release after our major push.
'), // rate_limit $this->getOKResponse(['resources' => ['core' => ['reset' => time() + 1000, 'limit' => 200, 'remaining' => 10]]]), ]); $clientHandler = HandlerStack::create($responses); $guzzleClient = new Client([ 'handler' => $clientHandler, ]); $httpClient = new Guzzle7Client($guzzleClient); $httpBuilder = new Builder($httpClient); $githubClient = new GithubClient($httpBuilder); $client = static::createClient(); // override factory to avoid real call to Github self::getContainer()->set('banditore.client.github.test', $githubClient); // mock pubsubhubbub request $guzzleClientPub = $this->getMockBuilder(Client::class) ->disableOriginalConstructor() ->getMock(); $guzzleClientPub->expects($this->once()) ->method('post') ->willReturn(new Response(204)); self::getContainer()->set('banditore.client.guzzle.test', $guzzleClientPub); $handler = self::getContainer()->get(VersionsSyncHandler::class); /** @var Version[] */ $versions = self::getContainer()->get(VersionRepository::class)->findBy(['repo' => 555]); $this->assertCount(1, $versions, 'Repo 555 has 1 version'); $this->assertSame('1.0.21', $versions[0]->getTagName(), 'Repo 555 has 1 version, which is 1.0.21'); $handler->__invoke(new VersionsSync(555)); /** @var Version[] */ $versions = self::getContainer()->get(VersionRepository::class)->findBy(['repo' => 555]); $this->assertCount(2, $versions, 'Repo 555 has now 2 versions'); $this->assertSame('1.0.21', $versions[0]->getTagName(), 'Repo 555 has 2 version. First one is 1.0.21'); $this->assertSame('V1.1.0', $versions[1]->getTagName(), 'Repo 555 has 2 version. Second one is V1.1.0'); $this->assertSame('This is the first release after our major push.
', $versions[1]->getBody(), 'Version V1.1.0 body is ok'); } public function testProcessSuccessfulMessageWithBlobTag(): void { $uow = $this->getMockBuilder(UnitOfWork::class) ->disableOriginalConstructor() ->getMock(); $uow->expects($this->once()) ->method('getScheduledEntityInsertions') ->willReturn([]); $em = $this->getMockBuilder(EntityManager::class) ->disableOriginalConstructor() ->getMock(); $em->expects($this->once()) ->method('isOpen') ->willReturn(false); // simulate a closing manager $em->expects($this->once()) ->method('getUnitOfWork') ->willReturn($uow); $doctrine = $this->getMockBuilder(Registry::class) ->disableOriginalConstructor() ->getMock(); $doctrine->expects($this->once()) ->method('getManager') ->willReturn($em); $doctrine->expects($this->once()) ->method('resetManager') ->willReturn($em); $repo = new Repo(); $repo->setId(123); $repo->setFullName('bob/wow'); $repo->setName('wow'); $repoRepository = $this->getMockBuilder(RepoRepository::class) ->disableOriginalConstructor() ->getMock(); $repoRepository->expects($this->once()) ->method('find') ->with(123) ->willReturn($repo); $versionRepository = $this->getMockBuilder(VersionRepository::class) ->disableOriginalConstructor() ->getMock(); $versionRepository->expects($this->once()) ->method('findExistingOne') ->willReturn(null); $pubsubhubbub = $this->getMockBuilder(Publisher::class) ->disableOriginalConstructor() ->getMock(); $pubsubhubbub->expects($this->once()) ->method('pingHub') ->with([123]); $responses = new MockHandler([ // rate_limit $this->getOKResponse(['resources' => ['core' => ['reset' => time() + 1000, 'limit' => 200, 'remaining' => 10]]]), // repo/tags $this->getOKResponse([[ 'name' => 'street/wilson_gardens', 'zipball_url' => 'https://api.github.com/repos/nivbend/gitstery/zipball/street/wilson_gardens', 'tarball_url' => 'https://api.github.com/repos/nivbend/gitstery/tarball/street/wilson_gardens', 'commit' => [ 'sha' => '659f0c110cd80286eaff33d34b9caf6c8e183102', 'url' => 'https://api.github.com/repos/nivbend/gitstery/commits/659f0c110cd80286eaff33d34b9caf6c8e183102', ], ]]), // git/refs/tags $this->getOKResponse([ [ 'ref' => 'refs/tags/solution', 'url' => 'https://api.github.com/repos/nivbend/gitstery/git/refs/tags/solution', 'object' => [ 'sha' => 'b3618a9ec1bbc13bf7133c50fb8d15ef8cbe7594', 'type' => 'blob', 'url' => 'https://api.github.com/repos/nivbend/gitstery/git/blobs/b3618a9ec1bbc13bf7133c50fb8d15ef8cbe7594', ], ], ]), // TAG solution // repos/release with tag solution (which is not a release) new Response(404, ['Content-Type' => 'application/json'], (string) json_encode([ 'message' => 'Not Found', 'documentation_url' => 'https://developer.github.com/v3', ])), // retrieve tag information from the blob $this->getOKResponse([ 'sha' => 'b3618a9ec1bbc13bf7133c50fb8d15ef8cbe7594', 'url' => 'https://api.github.com/repos/nivbend/gitstery/git/blobs/b3618a9ec1bbc13bf7133c50fb8d15ef8cbe7594', 'size' => 40, 'content' => "ZGUxMzI0OTUxYWZlNmU0NjI0MDY2MGNiYzAzYzE1MDBhOTBmYzkyOA==\n", 'encoding' => 'base64', ]), // markdown new Response(200, ['Content-Type' => 'text/html'], '(blob, size 40) de1324951afe6e46240660cbc03c1500a90fc928
'), // rate_limit $this->getOKResponse(['resources' => ['core' => ['reset' => time() + 1000, 'limit' => 200, 'remaining' => 10]]]), ]); $clientHandler = HandlerStack::create($responses); $guzzleClient = new Client([ 'handler' => $clientHandler, ]); $httpClient = new Guzzle7Client($guzzleClient); $httpBuilder = new Builder($httpClient); $githubClient = new GithubClient($httpBuilder); $logger = new Logger('foo'); $logHandler = new TestHandler(); $logger->pushHandler($logHandler); $handler = new VersionsSyncHandler( $doctrine, $repoRepository, $versionRepository, $pubsubhubbub, $githubClient, $logger ); $handler->__invoke(new VersionsSync(123)); $records = $logHandler->getRecords(); $this->assertSame('Consume banditore.sync_versions message', $records[0]['message']); $this->assertSame('[10] Checkyay
', 'createdAt' => (new \DateTime())->setTimestamp(1171502725), ], ], 'http://myfeed.api/.rss' ); $this->assertSame('New releases from starred repo of bob', $channel->getTitle()); $this->assertSame('http://myfeed.api/.rss', $channel->getLink()); $this->assertSame('Here are all the new releases from all repos starred by bob', $channel->getDescription()); $this->assertSame('en', $channel->getLanguage()); $this->assertStringContainsString('(c)', $channel->getCopyright()); $this->assertStringContainsString('banditore', $channel->getCopyright()); $this->assertStringContainsString('15 Feb 2007', $channel->getLastBuildDate()->format('r')); $this->assertSame('banditore', $channel->getGenerator()); $items = $channel->getItems(); $this->assertCount(1, $items); $this->assertSame('test/test 1.0.0', $items[0]->getTitle()); $this->assertSame('https://github.com/test/test/releases/1.0.0', $items[0]->getLink()); $this->assertStringContainsString('
', $items[0]->getDescription());
$this->assertStringContainsString('#Thus', $items[0]->getDescription());
$this->assertStringContainsString('yay
', $items[0]->getDescription()); $this->assertStringContainsString('test/test', $items[0]->getDescription()); $this->assertStringContainsString('(http://homepa.ge)', $items[0]->getDescription()); $this->assertStringContainsString('This is an awesome description', $items[0]->getDescription()); $this->assertSame('https://github.com/test/test/releases/1.0.0', $items[0]->getGuid()->getGuid()); $this->assertTrue($items[0]->getGuid()->getIsPermaLink()); $this->assertStringContainsString('15 Feb 2007', $items[0]->getPubDate()->format('r')); } } ================================================ FILE: tests/Security/GithubAuthenticatorTest.php ================================================ markTestSkipped('Dunno how to mock the session / access it from the container'); $client = static::createClient(); $responses = new MockHandler([ // /login/oauth/access_token (to retrieve the access_token from `authenticate()`) new Response(200, ['Content-Type' => 'application/json'], (string) json_encode([ 'access_token' => 'blablabla', ])), // /api/v3/user (to retrieve user information from Github) new Response(200, ['Content-Type' => 'application/json'], (string) json_encode([ 'id' => 123, 'email' => 'toto@test.io', 'name' => 'Bob', 'login' => 'admin', 'avatar_url' => 'http://avat.ar/my.png', ])), ]); $clientHandler = HandlerStack::create($responses); $guzzleClient = new Client([ 'handler' => $clientHandler, ]); $httpClient = new Guzzle7Client($guzzleClient); $httpBuilder = new Builder($httpClient); $githubClient = new GithubClient($httpBuilder); self::getContainer()->set('banditore.client.github.application', $githubClient); self::getContainer()->get('oauth2.registry')->getClient('github')->getOAuth2Provider()->setHttpClient($guzzleClient); self::getContainer()->get('session')->set(OAuth2Client::OAUTH2_SESSION_STATE_KEY, 'MyAwesomeState'); // before login /** @var User */ $user = self::getContainer()->get(UserRepository::class)->find(123); $this->assertSame('1234567890', $user->getAccessToken()); $this->assertSame('http://0.0.0.0/avatar.jpg', $user->getAvatar()); $client->request('GET', '/callback?state=MyAwesomeState&code=MyAwesomeCode'); // after login /** @var User */ $user = self::getContainer()->get(UserRepository::class)->find(123); $this->assertSame('blablabla', $user->getAccessToken()); $this->assertSame('http://avat.ar/my.png', $user->getAvatar()); $this->assertSame(302, $client->getResponse()->getStatusCode()); /** @var RedirectResponse */ $response = $client->getResponse(); $this->assertSame('/dashboard', $response->getTargetUrl()); $message = self::getContainer()->get('session')->getFlashBag()->get('info'); $this->assertSame('Successfully logged in!', $message[0]); $transport = self::getContainer()->get('messenger.transport.sync_starred_repos'); $this->assertCount(1, $transport->get()); $messages = (array) $transport->get(); /** @var StarredReposSync */ $message = $messages[0]->getMessage(); $this->assertSame(123, $message->getUserId()); } public function testCallbackWithNewUser(): void { $this->markTestSkipped('Dunno how to mock the session / access it from the container'); $client = static::createClient(); $responses = new MockHandler([ // /login/oauth/access_token (to retrieve the access_token from `authenticate()`) new Response(200, ['Content-Type' => 'application/json'], (string) json_encode([ 'access_token' => 'superboum', ])), // /api/v3/user (to retrieve user information from Github) new Response(200, ['Content-Type' => 'application/json'], (string) json_encode([ 'id' => 456, 'email' => 'down@g.et', 'name' => 'Any', 'login' => 'getdown', 'avatar_url' => 'http://avat.ar/down.png', ])), ]); $clientHandler = HandlerStack::create($responses); $guzzleClient = new Client([ 'handler' => $clientHandler, ]); $httpClient = new Guzzle7Client($guzzleClient); $httpBuilder = new Builder($httpClient); $githubClient = new GithubClient($httpBuilder); self::getContainer()->set('banditore.client.github.application', $githubClient); self::getContainer()->get('oauth2.registry')->getClient('github')->getOAuth2Provider()->setHttpClient($guzzleClient); self::getContainer()->get('session')->set(OAuth2Client::OAUTH2_SESSION_STATE_KEY, 'MyAwesomeState'); // before login $user = self::getContainer()->get(UserRepository::class)->find(456); $this->assertNull($user, 'User 456 does not YET exist'); $client->request('GET', '/callback?state=MyAwesomeState&code=MyAwesomeCode'); // after login /** @var User */ $user = self::getContainer()->get(UserRepository::class)->find(456); $this->assertSame('superboum', $user->getAccessToken()); $this->assertSame('http://avat.ar/down.png', $user->getAvatar()); $this->assertSame('getdown', $user->getUsername()); $this->assertSame('Any', $user->getName()); $this->assertSame(302, $client->getResponse()->getStatusCode()); /** @var RedirectResponse */ $response = $client->getResponse(); $this->assertSame('/dashboard', $response->getTargetUrl()); $message = self::getContainer()->get('session')->getFlashBag()->get('info'); $this->assertSame('Successfully logged in. Your starred repos will soon be synced!', $message[0]); $transport = self::getContainer()->get('messenger.transport.sync_starred_repos'); $this->assertCount(1, $transport->get()); $messages = (array) $transport->get(); /** @var StarredReposSync */ $message = $messages[0]->getMessage(); $this->assertSame(456, $message->getUserId()); } } ================================================ FILE: tests/Twig/RepoVersionExtensionTest.php ================================================ assertNull($ext->linkToVersion([])); $this->assertNull($ext->linkToVersion(['fullName' => 'test/test'])); $this->assertNull($ext->linkToVersion(['tagName' => 'v1.0.0'])); $this->assertSame('https://github.com/test/test/releases/v1.0.0', $ext->linkToVersion(['fullName' => 'test/test', 'tagName' => 'v1.0.0'])); } public function testEncodedTagName(): void { $ext = new RepoVersionExtension(); $this->assertNull($ext->linkToVersion([])); $this->assertNull($ext->linkToVersion(['fullName' => 'test/test'])); $this->assertNull($ext->linkToVersion(['tagName' => '@1.0.0-alpha.1'])); $this->assertSame('https://github.com/test/test/releases/%401.0.0-alpha.1', $ext->linkToVersion(['fullName' => 'test/test', 'tagName' => '@1.0.0-alpha.1'])); } } ================================================ FILE: tests/Webfeeds/WebfeedsTest.php ================================================ setLogo('https://upload.wikimedia.org/wikipedia/commons/a/ab/Logo_TV_2015.png') ->setIcon('https://upload.wikimedia.org/wikipedia/commons/a/ab/Logo_TV_2015.png') ->setAccentColor('404040'); $this->assertSame('https://upload.wikimedia.org/wikipedia/commons/a/ab/Logo_TV_2015.png', $webfeeds->getLogo()); $this->assertSame('https://upload.wikimedia.org/wikipedia/commons/a/ab/Logo_TV_2015.png', $webfeeds->getIcon()); $this->assertSame('404040', $webfeeds->getAccentColor()); } } ================================================ FILE: tests/Webfeeds/WebfeedsWriterTest.php ================================================ setLogo('https://upload.wikimedia.org/wikipedia/commons/a/ab/Logo_TV_2015.png') ->setIcon('https://upload.wikimedia.org/wikipedia/commons/a/ab/Logo_TV_2015.png') ->setAccentColor('404040'); $writer->write($rssWriter, $webfeeds); $expected = <<<'EOF'