Repository: Mohammedcha/gplay-scraper
Branch: main
Commit: 304dcd3d3546
Files: 81
Total size: 567.4 KB
Directory structure:
gitextract_ihudd3jc/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ └── feature_request.md
│ ├── pull_request_template.md
│ └── workflows/
│ ├── docs.yml
│ └── test.yml
├── .gitignore
├── CHANGELOG.md
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── MANIFEST.in
├── README/
│ ├── APP_METHODS.md
│ ├── DEVELOPER_METHODS.md
│ ├── LIST_METHODS.md
│ ├── README.md
│ ├── REVIEWS_METHODS.md
│ ├── SEARCH_METHODS.md
│ ├── SIMILAR_METHODS.md
│ └── SUGGEST_METHODS.md
├── README.md
├── SECURITY.md
├── build_docs.py
├── docs/
│ ├── README.md
│ ├── api/
│ │ ├── app.rst
│ │ ├── developer.rst
│ │ ├── list.rst
│ │ ├── reviews.rst
│ │ ├── search.rst
│ │ ├── similar.rst
│ │ └── suggest.rst
│ ├── conf.py
│ ├── configuration.rst
│ ├── error_handling.rst
│ ├── examples.rst
│ ├── fields.rst
│ ├── index.rst
│ ├── installation.rst
│ ├── quickstart.rst
│ └── requirements.txt
├── examples/
│ ├── README.md
│ ├── app_methods_example.py
│ ├── developer_methods_example.py
│ ├── list_methods_example.py
│ ├── reviews_methods_example.py
│ ├── search_methods_example.py
│ ├── similar_methods_example.py
│ └── suggest_methods_example.py
├── gplay_scraper/
│ ├── __init__.py
│ ├── app.py
│ ├── config.py
│ ├── core/
│ │ ├── __init__.py
│ │ ├── gplay_methods.py
│ │ ├── gplay_parser.py
│ │ └── gplay_scraper.py
│ ├── exceptions.py
│ ├── models/
│ │ ├── __init__.py
│ │ └── element_specs.py
│ └── utils/
│ ├── __init__.py
│ ├── constants.py
│ ├── error_handling.py
│ ├── helpers.py
│ └── http_client.py
├── output/
│ ├── app_example.json
│ ├── developer_example.json
│ ├── list_example.json
│ ├── reviews_example.json
│ ├── search_example.json
│ ├── similar_example.json
│ └── suggest_example.json
├── requirements.txt
├── setup.py
└── tests/
├── __init__.py
├── test_app_methods.py
├── test_basic.py
├── test_developer_methods.py
├── test_list_methods.py
├── test_package.py
├── test_reviews_methods.py
├── test_search_methods.py
├── test_similar_methods.py
└── test_suggest_methods.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: Bug report
about: Create a report to help us improve
title: '[BUG] '
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Use app ID '...'
2. Call method '....'
3. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Code Example**
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Your code here
```
**Error Output**
```
Paste the full error message here
```
**Environment:**
- OS: [e.g. Windows 10, macOS, Linux]
- Python version: [e.g. 3.8.5]
- Library version: [e.g. 1.0.2]
**Additional context**
Add any other context about the problem here.
================================================
FILE: .github/ISSUE_TEMPLATE/feature_request.md
================================================
---
name: Feature request
about: Suggest an idea for this project
title: '[FEATURE] '
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Use Case**
Describe how this feature would be used:
```python
# Example of how the new feature would work
scraper = GPlayScraper()
result = scraper.new_method(app_id)
```
**Additional context**
Add any other context or screenshots about the feature request here.
================================================
FILE: .github/pull_request_template.md
================================================
# Pull Request
## Description
Brief description of changes made.
## Type of Change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation update
## Testing
- [ ] I have tested my changes locally
- [ ] I have added tests for new functionality
- [ ] All existing tests pass
## Code Quality
- [ ] My code follows the project's style guidelines
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
## Related Issues
Fixes #(issue number)
## Additional Notes
Any additional information about the changes.
================================================
FILE: .github/workflows/docs.yml
================================================
name: Build and Deploy Documentation
on:
push:
branches: [ main ]
permissions:
contents: write
jobs:
docs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r docs/requirements.txt
- name: Build documentation
run: |
cd docs
sphinx-build -b html . _build/html
touch _build/html/.nojekyll
- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v4
if: github.ref == 'refs/heads/main'
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs/_build/html
force_orphan: true
================================================
FILE: .github/workflows/test.yml
================================================
name: Tests
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pytest pytest-cov
- name: Run package and basic functionality tests
run: |
python -m unittest tests.test_package tests.test_basic -v
- name: Run network-dependent tests (optional)
continue-on-error: true
timeout-minutes: 15
run: |
echo "Running network-dependent tests with delays (failures expected due to rate limiting)..."
python -m unittest tests.test_app_methods -v || echo "App methods test completed"
python -m unittest tests.test_search_methods -v || echo "Search methods test completed"
python -m unittest tests.test_reviews_methods -v || echo "Reviews methods test completed"
python -m unittest tests.test_developer_methods -v || echo "Developer methods test completed"
python -m unittest tests.test_list_methods -v || echo "List methods test completed"
python -m unittest tests.test_similar_methods -v || echo "Similar methods test completed"
python -m unittest tests.test_suggest_methods -v || echo "Suggest methods test completed"
================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
!docs/.nojekyll
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
Pipfile.lock
# PEP 582
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# IDE
.vscode/
.idea/
*.swp
*.swo
# OS
.DS_Store
Thumbs.db
# Project specific
backup/
temp/
*.tmp
# Publishing scripts (local use only)
publish_to_github.bat
publish_to_github.sh
publish_to_pypi.bat
publish_to_pypi.sh
update_github.bat
update_github.sh
update_pypi.bat
update_pypi.sh
# Test and debug files
xx.py
x_fallback.py
test_all_methods.py
debug_limbo*.py
limbo_ds5_raw.txt
appbrain_scraper.py
# Documentation folders (local use only)
wiki/
community/
# Chrome extensions
chrome-extension/
chrome-extension-new/
firefox-extensions/
firefox-extension-new/
edge-extensions/
edge-extension-new/
opera-extensions/
opera-extension-new/
webstore-upload/
webstore-upload-new/
webstore-upload-chrome/
webstore-upload-firefox/
webstore-upload-edge/
webstore-upload-opera/
webstore-upload-chrome-new/
webstore-upload-firefox-new/
webstore-upload-edge-new/
webstore-upload-opera-new/
build-extensions/
build-extension/
build-extension-chrome/
build-extension-firefox/
build-extension-edge/
build-extension-opera/
build-extension-chrome-new/
build-extension-firefox-new/
build-extension-edge-new/
build-extension-opera-new/
dist-extensions/
dist-extension/
dist-extension-chrome/
dist-extension-firefox/
dist-extension-edge/
dist-extension-opera/
dist-extension-chrome-new/
dist-extension-firefox-new/
dist-extension-edge-new/
dist-extension-opera-new/
release/
release-chrome/
release-firefox/
release-edge/
release-opera/
release-chrome-new/
release-firefox-new/
release-edge-new/
release-opera-new/
temp-extensions/
temp-extension/
temp-extension-chrome/
temp-extension-firefox/
temp-extension-edge/
temp-extension-opera/
temp-extension-chrome-new/
================================================
FILE: CHANGELOG.md
================================================
# Changelog
All notable changes to this project will be documented in this file.
## [1.0.6] - 2025-11-16
### Bug Fixes
- **Reviews Pagination Fix**: Fixed critical issue when requesting more reviews than available
- Resolved 'NoneType' object is not subscriptable error
- Improved token extraction logic for empty review responses
- Now gracefully returns available reviews instead of crashing
- Enhanced error handling in ReviewsScraper and ReviewsParser
- **Empty Response Handling**: Better handling of apps with limited reviews
- Safe bounds checking for pagination tokens
- Proper null checking for empty data structures
- Graceful degradation when no more reviews are available
### Acknowledgments
- Thanks to [@PhamDinhThienVu](https://github.com/PhamDinhThienVu) for reporting the reviews pagination bug
## [1.0.5] - 2025-10-18
### New Features
- **Publisher Country Detection**: Added `publisherCountry` field to app data
- Automatically detects developer's country from phone number and address
- Uses international phone prefixes and address parsing
- Returns country names like "United States", "Germany", "Japan", etc.
- Handles multiple countries when phone and address differ (e.g., "United States/Germany")
### Removed Features
- **Removed updatedTimestamp**: Removed deprecated timestamp field that was causing confusion
### Bug Fixes
- **Enhanced Error Handling**: Improved error handling and retry mechanisms
- Better HTTP client fallback when requests fail
- More robust JSON parsing with multiple fallback strategies
- Improved handling of network timeouts and connection errors
- **Retry Mechanism**: Fixed automatic retry logic for failed requests
- Exponential backoff for rate limiting
- Automatic HTTP client switching on failures
- Better error recovery for temporary network issues
- **General Bug Fixes**: Fixed various edge cases and improved stability
- Better handling of malformed JSON responses
- Improved data extraction for apps with missing fields
- Enhanced Unicode handling for international app data
## [1.0.4] - 2025-10-16
### New Features
- **Assets Parameter**: Added configurable image sizes for all app methods
- `SMALL` (512px width)
- `MEDIUM` (1024px width) - Default
- `LARGE` (2048px width)
- `ORIGINAL` (Maximum size)
- Available in all app methods: `app_analyze()`, `app_get_field()`, `app_get_fields()`, `app_print_field()`, `app_print_fields()`, `app_print_all()`
- Affects icon, headerImage, screenshots, and videoImage URLs
### Bug Fixes
- **Release Date Fallback**: Fixed missing release dates when using language/country parameters
- Added automatic fallback request without `hl`/`gl` parameters when release date is null
- Ensures release date extraction for apps in all regions
- **Path Resolution**: Fixed various path-related issues in data extraction
- **Image URL Processing**: Improved image URL formatting with proper size parameters
### Usage Examples
```python
# Use different asset sizes
data = scraper.app_analyze("com.whatsapp", assets="LARGE")
icon = scraper.app_get_field("com.whatsapp", "icon", assets="SMALL")
scraper.app_print_all("com.whatsapp", assets="ORIGINAL")
```
## [1.0.3] - 2025-10-15
### New Features
- **Enhanced Search Pagination**: Now able to fetch unlimited search results (300+) with automatic pagination, not limited to 50 results anymore
- **Improved Search Performance**: Optimized search result fetching with better token handling and batch processing
### Bug Fixes & Code Quality Improvements
- **Code Review**: Addressed security vulnerabilities and code quality issues
- **Error Handling**: Improved error handling patterns across all modules
- **Performance**: Optimized JSON parsing and HTTP client fallback logic
- **Security**: Fixed potential SSRF and injection vulnerabilities
- **Maintainability**: Enhanced code readability and documentation
## [1.0.2] - 2025-01-15
### Major Release - Complete Library Redesign 🚀
This version represents a complete rewrite of GPlay Scraper with a focus on modularity, extensibility, and comprehensive data extraction across all Google Play Store features.
### New Features
#### 7 Method Types with 42 Functions
- **App Methods** - Extract 65+ data fields from any app (ratings, installs, pricing, permissions, screenshots, etc.)
- **Search Methods** - Search Google Play Store apps with comprehensive filtering and pagination
- **Reviews Methods** - Extract user reviews with ratings, timestamps, helpful votes, and detailed feedback
- **Developer Methods** - Get all apps published by a specific developer using developer ID
- **List Methods** - Access top charts (TOP_FREE, TOP_PAID, TOP_GROSSING) by category with 54 categories
- **Similar Methods** - Find similar/competitor apps for market research and competitive analysis
- **Suggest Methods** - Get search suggestions and autocomplete for ASO keyword research
Each method type includes 6 functions:
- `analyze()` - Get all data as dictionary/list
- `get_field()` - Get single field value
- `get_fields()` - Get multiple fields as dictionary
- `print_field()` - Print single field to console
- `print_fields()` - Print multiple fields to console
- `print_all()` - Print all data as formatted JSON
#### 7 HTTP Clients with Automatic Fallback
- **requests** (default) - Standard Python HTTP library, reliable and well-tested
- **curl_cffi** - Browser impersonation with TLS fingerprinting, best for avoiding detection
- **tls_client** - Custom TLS fingerprinting, good for bypassing restrictions
- **httpx** - Modern async-capable HTTP client with HTTP/2 support
- **urllib3** - Low-level HTTP client with connection pooling
- **cloudscraper** - Cloudflare bypass capabilities
- **aiohttp** - Async HTTP client for high-performance concurrent requests
Automatic fallback system tries clients in order until one succeeds, ensuring maximum reliability.
#### Multi-Language & Multi-Region Support
- Support for 100+ languages (en, es, fr, de, ja, ko, zh, ar, etc.)
- Support for 150+ countries (us, gb, ca, au, in, br, jp, etc.)
- Get localized app data, reviews, and search results
- Region-specific pricing and availability information
#### Comprehensive Data Extraction
- **65+ App Fields**: title, developer, ratings, installs, price, screenshots, permissions, release date, update date, size, version, content rating, privacy policy, and more
- **Review Data**: user name, rating, review text, timestamp, app version, helpful votes, developer reply
- **Search Results**: app ID, title, developer, rating, price, icon, screenshots, description snippet
- **Developer Portfolio**: all apps from a developer with complete metadata
- **Top Charts**: ranked lists with install counts, ratings, and trending data
- **Similar Apps**: competitor analysis with relevance scoring
- **Search Suggestions**: popular keywords and autocomplete terms
#### Enhanced Architecture
- **Modular Design**: Separate classes for methods, scrapers, and parsers
- **Core Modules**: `gplay_methods.py`, `gplay_scraper.py`, `gplay_parser.py`
- **HTTP Client Abstraction**: `HttpClient` class with pluggable client support
- **Element Specs**: Reusable CSS selector specifications for data extraction
- **Helper Utilities**: Text processing, date parsing, JSON cleaning, age calculation
- **Exception Hierarchy**: 6 custom exception types for specific error scenarios
#### Documentation & Testing
- **Comprehensive Docstrings**: All 42 methods, 7 scrapers, 7 parsers, and utility functions documented
- **Sphinx Documentation**: Professional HTML documentation with examples, API reference, and guides
- **HTTP Clients Guide**: Detailed documentation on when and how to use each HTTP client
- **Fields Reference**: Complete reference of all 65+ fields, categories, and parameters
- **Unit Tests**: Complete test coverage for all 7 method types
- **Examples**: Real-world usage examples for each method type
#### Configuration & Customization
- **Configurable Parameters**: Language, country, count, sort order, collection type
- **Rate Limiting**: Built-in delays to prevent blocking (configurable)
- **Error Handling**: Graceful fallbacks and informative error messages
- **Logging**: Detailed logging for debugging and monitoring
- **Timeout Control**: Configurable request timeouts
- **Retry Logic**: Automatic retries with exponential backoff
### Breaking Changes
- Complete API redesign - not backward compatible with v1.0.1
- Method names changed from `get_app_details()` to `app_analyze()`
- New parameter structure for all methods
- HTTP client must be specified or uses automatic fallback
- Exception types renamed and reorganized
### Migration Guide
Old (v1.0.1):
```python
scraper = GPlayScraper()
data = scraper.get_app_details("com.whatsapp")
```
New (v1.0.2):
```python
scraper = GPlayScraper()
data = scraper.app_analyze("com.whatsapp")
```
### Performance Improvements
- Faster JSON parsing with optimized regex patterns
- Reduced memory usage with streaming parsers
- Better caching of HTTP client instances
- Parallel request support with async clients
### Bug Fixes
- Fixed JSON parsing for apps with special characters in descriptions
- Fixed review extraction for apps with no reviews
- Fixed developer ID extraction from developer pages
- Fixed category parsing for apps in multiple categories
- Fixed price parsing for apps with regional pricing
- Fixed screenshot URL extraction for apps with video previews
## [1.0.1] - 2025-10-07
### Added
- **Paid App Support**: Fixed JSON parsing issues for paid apps with malformed data structures
- **Reviews Extraction**: Successfully extracts user reviews for both free and paid apps
- **Organized Output**: Restructured JSON output with logical field grouping:
- Basic Information
- Category & Genre
- Release & Updates
- Media Content
- Install Statistics
- Ratings & Reviews
- Advertising
- Technical Details
- Content Rating
- Privacy & Security
- Pricing & Monetization
- Developer Information
- ASO Analysis
- **Enhanced JSON Parser**: Bracket-matching algorithm for complex nested structures
- **Original Price Field**: Added `originalPrice` field for sale price tracking
### Fixed
- **JSON Parsing Errors**: Resolved "Expecting ',' delimiter" errors for paid apps
- **Reviews Data**: Fixed empty reviews arrays by implementing alternative parsing methods
- **Malformed Data Handling**: Improved handling of unquoted keys and malformed JSON from Play Store
### Improved
- **Error Handling**: Better fallback mechanisms for JSON parsing failures
- **Data Extraction**: More robust extraction for apps with complex pricing structures
- **Code Organization**: Cleaner separation of parsing logic and error recovery
## [1.0.0] - 2025-10-06
### Added
- Initial release of GPlay Scraper
- Complete Google Play Store app data extraction
- ASO (App Store Optimization) analysis
- Modular architecture with separate core modules
- Support for 60+ data fields including:
- Basic app information
- Install statistics and metrics
- Ratings and reviews data
- Technical specifications
- Developer information
- Media content (screenshots, videos, icons)
- Pricing and monetization details
- ASO keyword analysis
- Multiple access methods:
- `analyze()` - Complete app analysis
- `get_field()` - Single field retrieval
- `get_fields()` - Multiple field retrieval
- `print_field()` - Direct field printing
- `print_fields()` - Multiple field printing
- `print_all()` - Complete data printing
- Comprehensive documentation and examples
- Error handling and logging
- Rate limiting considerations
- Cross-platform compatibility
### Features
- Web scraping of Google Play Store pages
- JSON data extraction and parsing
- Automatic install metrics calculation
- Keyword frequency analysis
- Readability scoring
- Review data extraction
- Image URL processing
- Date parsing and age calculation
================================================
FILE: CODE_OF_CONDUCT.md
================================================
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement through GitHub Issues.
All complaints will be reviewed and investigated promptly and fairly.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
[homepage]: https://www.contributor-covenant.org
================================================
FILE: CONTRIBUTING.md
================================================
# Contributing to GPlay Scraper
Thank you for your interest in contributing!
## Development Setup
1. Fork the repository
2. Clone your fork: `git clone https://github.com/yourusername/gplay-scraper.git`
3. Install in development mode: `pip install -e .`
4. Install dev dependencies: `pip install pytest`
## Running Tests
```bash
python -m pytest tests/ -v
```
## Code Style
- Follow PEP 8
- Add docstrings to new functions
- Include type hints where appropriate
## Submitting Changes
1. Create a feature branch: `git checkout -b feature-name`
2. Make your changes
3. Add tests for new functionality
4. Run tests to ensure they pass
5. Submit a pull request
## Reporting Issues
Please use GitHub Issues to report bugs or request features.
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2025 Mohammed Cha
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: MANIFEST.in
================================================
include README.md
include LICENSE
include requirements.txt
include CHANGELOG.md
include CONTRIBUTING.md
include SECURITY.md
recursive-include examples *.py
recursive-include tests *.py
================================================
FILE: README/APP_METHODS.md
================================================
# App Methods
Extract detailed information about individual Google Play Store apps.
## Quick Start
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Get all data
data = scraper.app_analyze("com.whatsapp")
print(data['title'], data['score'], data['installs'])
# Get specific fields
title = scraper.app_get_field("com.whatsapp", "title")
print(title) # WhatsApp Messenger
# Get multiple fields
info = scraper.app_get_fields("com.whatsapp", ["title", "score", "developer"])
print(info)
```
---
## HTTP Clients
The library supports 7 HTTP clients with automatic fallback. If one fails, it tries the next.
### Supported Clients
1. **requests** (default) - Standard Python HTTP library
2. **curl_cffi** - cURL with browser impersonation
3. **tls_client** - Advanced TLS fingerprinting
4. **httpx** - Modern async-capable HTTP client
5. **urllib3** - Low-level HTTP client
6. **cloudscraper** - Cloudflare bypass
7. **aiohttp** - Async HTTP client
### Usage
```python
# Default (tries requests first, then others)
scraper = GPlayScraper()
# Specify a client
scraper = GPlayScraper(http_client="curl_cffi")
scraper = GPlayScraper(http_client="tls_client")
scraper = GPlayScraper(http_client="httpx")
```
### Installation
```bash
# Default
pip install requests
# Advanced clients (optional)
pip install curl-cffi
pip install tls-client
pip install httpx
pip install urllib3
pip install cloudscraper
pip install aiohttp
```
**Note:** The library automatically falls back to available clients if your preferred one fails.
---
## Methods
### `app_analyze(app_id, lang='en', country='us', assets=None)`
Returns all 65+ fields as a dictionary.
```python
data = scraper.app_analyze("com.whatsapp")
# Returns: {'appId': 'com.whatsapp', 'title': 'WhatsApp Messenger', ...}
# With custom image sizes
data = scraper.app_analyze("com.whatsapp", assets="LARGE")
# Returns same data but with larger image URLs (2048px)
```
### `app_get_field(app_id, field, lang='en', country='us', assets=None)`
Returns a single field value.
```python
score = scraper.app_get_field("com.whatsapp", "score")
# Returns: 4.2
# Get high-quality icon
icon = scraper.app_get_field("com.whatsapp", "icon", assets="ORIGINAL")
# Returns: URL with maximum image quality
```
### `app_get_fields(app_id, fields, lang='en', country='us', assets=None)`
Returns multiple fields as a dictionary.
```python
data = scraper.app_get_fields("com.whatsapp", ["title", "score", "installs"])
# Returns: {'title': 'WhatsApp Messenger', 'score': 4.2, 'installs': '5,000,000,000+'}
# Get media with custom sizes
media = scraper.app_get_fields("com.whatsapp", ["icon", "screenshots"], assets="SMALL")
# Returns: Media URLs with 512px width
```
### `app_print_field(app_id, field, lang='en', country='us', assets=None)`
Prints a single field to console.
```python
scraper.app_print_field("com.whatsapp", "title")
# Output: title: WhatsApp Messenger
# Print large icon URL
scraper.app_print_field("com.whatsapp", "icon", assets="LARGE")
# Output: icon: https://...=w2048
```
### `app_print_fields(app_id, fields, lang='en', country='us', assets=None)`
Prints multiple fields to console.
```python
scraper.app_print_fields("com.whatsapp", ["title", "score"])
# Output:
# title: WhatsApp Messenger
# score: 4.2
# Print media with original quality
scraper.app_print_fields("com.whatsapp", ["icon", "screenshots"], assets="ORIGINAL")
# Output: URLs with maximum image quality
```
### `app_print_all(app_id, lang='en', country='us', assets=None)`
Prints all fields as formatted JSON.
```python
scraper.app_print_all("com.whatsapp")
# Output: Full JSON with all 65+ fields
# Print with high-quality images
scraper.app_print_all("com.whatsapp", assets="LARGE")
# Output: Full JSON with 2048px image URLs
```
---
## Available Fields (65+)
### Basic Information
- `appId` - Package name (e.g., "com.whatsapp")
- `title` - App name
- `summary` - Short description
- `description` - Full description
- `appUrl` - Play Store URL
### Ratings & Reviews
- `score` - Average rating (1-5)
- `ratings` - Total number of ratings
- `reviews` - Total number of reviews
- `histogram` - Rating distribution [1★, 2★, 3★, 4★, 5★]
### Install Metrics
- `installs` - Install range (e.g., "10,000,000+")
- `minInstalls` - Minimum installs
- `realInstalls` - Estimated real installs
- `dailyInstalls` - Estimated daily installs
- `monthlyInstalls` - Estimated monthly installs
- `minDailyInstalls` - Minimum daily installs
- `realDailyInstalls` - Real estimated daily installs
- `minMonthlyInstalls` - Minimum monthly installs
- `realMonthlyInstalls` - Real estimated monthly installs
### Pricing
- `price` - Price in currency (0 if free)
- `currency` - Currency code (e.g., "USD")
- `free` - Boolean, true if free
- `offersIAP` - Has in-app purchases
- `inAppProductPrice` - IAP price range
- `sale` - Currently on sale
- `originalPrice` - Original price if on sale
### Media
- `icon` - App icon URL
- `headerImage` - Header image URL
- `screenshots` - List of screenshot URLs
- `video` - Promo video URL
- `videoImage` - Video thumbnail URL
### Developer
- `developer` - Developer name
- `developerId` - Developer ID
- `developerEmail` - Contact email
- `developerWebsite` - Website URL
- `developerAddress` - Physical address
- `developerPhone` - Contact phone
- `privacyPolicy` - Privacy policy URL
- `publisherCountry` - Developer's country
### Category
- `genre` - Primary category (e.g., "Communication")
- `genreId` - Category ID (e.g., "COMMUNICATION")
- `categories` - List of categories
### Technical
- `version` - Current version
- `androidVersion` - Required Android version
- `minAndroidApi` - Minimum API level
- `maxAndroidApi` - Maximum API level
- `appBundle` - App bundle name
### Dates
- `released` - Release date (e.g., "Feb 24, 2009")
- `appAgeDays` - Age in days
- `lastUpdated` - Last update date
### Content
- `contentRating` - Age rating (e.g., "Everyone")
- `contentRatingDescription` - Rating description
- `whatsNew` - Recent changes list
- `permissions` - Required permissions dict
- `dataSafety` - Data safety info list
### Advertising
- `adSupported` - Contains ads
- `containsAds` - Shows advertisements
### Availability
- `available` - App is available
---
## Practical Examples
### Competitive Analysis
```python
apps = ["com.whatsapp", "com.telegram", "com.viber"]
for app_id in apps:
data = scraper.app_get_fields(app_id, ["title", "score", "realInstalls"])
print(f"{data['title']}: {data['score']}★ - {data['realInstalls']:,} installs")
```
### Monitor App Updates
```python
app_id = "com.whatsapp"
data = scraper.app_get_fields(app_id, ["version", "lastUpdated", "whatsNew"])
print(f"Version: {data['version']}")
print(f"Updated: {data['lastUpdated']}")
print(f"Changes: {data['whatsNew']}")
```
### Extract Developer Info
```python
app_id = "com.whatsapp"
dev_info = scraper.app_get_fields(app_id, [
"developer", "developerEmail", "developerWebsite"
])
print(dev_info)
```
### Get High-Quality Media
```python
app_id = "com.whatsapp"
# Get original quality images
media = scraper.app_get_fields(app_id, ["icon", "screenshots"], assets="ORIGINAL")
print(f"Icon: {media['icon']}") # Maximum quality
print(f"Screenshots: {len(media['screenshots'])} images")
# Get small thumbnails for faster loading
thumbnails = scraper.app_get_fields(app_id, ["icon", "headerImage"], assets="SMALL")
print(f"Small icon: {thumbnails['icon']}") # 512px
```
### Check Monetization
```python
app_id = "com.whatsapp"
money = scraper.app_get_fields(app_id, [
"free", "price", "offersIAP", "containsAds"
])
print(f"Free: {money['free']}")
print(f"Has IAP: {money['offersIAP']}")
print(f"Has Ads: {money['containsAds']}")
```
---
## Parameters
### Initialization
- `http_client` (str, optional) - HTTP client to use: "requests", "curl_cffi", "tls_client", "httpx", "urllib3", "cloudscraper", "aiohttp" (default: "requests")
### Method Parameters
- `app_id` (str, required) - App package name from Play Store URL
- `lang` (str, optional) - Language code (default: 'en')
- `country` (str, optional) - Country code (default: 'us')
- `assets` (str, optional) - Image size: 'SMALL', 'MEDIUM', 'LARGE', 'ORIGINAL' (default: 'MEDIUM')
- `field` (str) - Single field name
- `fields` (List[str]) - List of field names
### Assets Parameter (Image Sizes)
- **SMALL** - 512px width (`w512`)
- **MEDIUM** - 1024px width (`w1024`) - Default
- **LARGE** - 2048px width (`w2048`)
- **ORIGINAL** - Maximum size (`w9999`)
Affects these fields: `icon`, `headerImage`, `screenshots`, `videoImage`
```python
# Different image qualities
small_icon = scraper.app_get_field("com.whatsapp", "icon", assets="SMALL")
# Returns: https://...=w512
large_icon = scraper.app_get_field("com.whatsapp", "icon", assets="LARGE")
# Returns: https://...=w2048
original_icon = scraper.app_get_field("com.whatsapp", "icon", assets="ORIGINAL")
# Returns: https://...=w9999
```
### Finding App IDs
From Play Store URL: `https://play.google.com/store/apps/details?id=com.whatsapp`
The app_id is: `com.whatsapp`
### Language & Country Codes
- **Language**: 'en', 'es', 'fr', 'de', 'ja', 'ko', 'pt', 'ru', 'zh', etc.
- **Country**: 'us', 'gb', 'ca', 'au', 'in', 'br', 'jp', 'kr', 'de', 'fr', etc.
---
## When to Use Each Method
- **`app_analyze()`** - Need all data for comprehensive analysis
- **`app_get_field()`** - Need just one specific value
- **`app_get_fields()`** - Need several specific fields (more efficient than multiple get_field calls)
- **`app_print_field()`** - Quick debugging/console output
- **`app_print_fields()`** - Quick debugging of multiple values
- **`app_print_all()`** - Explore available data structure
---
## Advanced Features
### Rate Limiting
Built-in rate limiting (1 second delay between requests) prevents blocking.
### Error Handling
```python
from gplay_scraper import GPlayScraper, AppNotFoundError, NetworkError
scraper = GPlayScraper()
try:
data = scraper.app_analyze("invalid.app.id")
except AppNotFoundError:
print("App not found")
except NetworkError:
print("Network error occurred")
```
### Multi-Region Data
```python
# Get data from different regions
us_data = scraper.app_analyze("com.whatsapp", country="us")
uk_data = scraper.app_analyze("com.whatsapp", country="gb")
jp_data = scraper.app_analyze("com.whatsapp", country="jp", lang="ja")
```
================================================
FILE: README/DEVELOPER_METHODS.md
================================================
# Developer Methods
Get all apps published by a specific developer on Google Play Store.
## Quick Start
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Get all apps from a developer
apps = scraper.developer_analyze("5700313618786177705")
for app in apps:
print(f"{app['title']}: {app['score']}★")
# Get specific fields
titles = scraper.developer_get_field("5700313618786177705", "title")
print(titles)
# Get multiple fields
apps = scraper.developer_get_fields("5700313618786177705", ["title", "score", "free"])
print(apps)
```
---
## HTTP Clients
The library supports 7 HTTP clients with automatic fallback. If one fails, it tries the next.
### Supported Clients
1. **requests** (default) - Standard Python HTTP library
2. **curl_cffi** - cURL with browser impersonation
3. **tls_client** - Advanced TLS fingerprinting
4. **httpx** - Modern async-capable HTTP client
5. **urllib3** - Low-level HTTP client
6. **cloudscraper** - Cloudflare bypass
7. **aiohttp** - Async HTTP client
### Usage
```python
# Default (tries requests first, then others)
scraper = GPlayScraper()
# Specify a client
scraper = GPlayScraper(http_client="curl_cffi")
scraper = GPlayScraper(http_client="tls_client")
scraper = GPlayScraper(http_client="httpx")
```
### Installation
```bash
# Default
pip install requests
# Advanced clients (optional)
pip install curl-cffi
pip install tls-client
pip install httpx
pip install urllib3
pip install cloudscraper
pip install aiohttp
```
**Note:** The library automatically falls back to available clients if your preferred one fails.
---
## Methods
### `developer_analyze(dev_id, count=100, lang='en', country='us')`
Returns all apps from a developer as a list of dictionaries.
```python
apps = scraper.developer_analyze("5700313618786177705", count=50)
# Returns: [{'appId': '...', 'title': '...', 'score': 4.5, ...}, ...]
```
### `developer_get_field(dev_id, field, count=100, lang='en', country='us')`
Returns a specific field from all developer apps.
```python
titles = scraper.developer_get_field("5700313618786177705", "title")
# Returns: ['App 1', 'App 2', 'App 3', ...]
```
### `developer_get_fields(dev_id, fields, count=100, lang='en', country='us')`
Returns multiple fields from all developer apps.
```python
apps = scraper.developer_get_fields("5700313618786177705", ["title", "score", "free"])
# Returns: [{'title': 'App 1', 'score': 4.5, 'free': True}, ...]
```
### `developer_print_field(dev_id, field, count=100, lang='en', country='us')`
Prints a specific field from all developer apps.
```python
scraper.developer_print_field("5700313618786177705", "title")
# Output:
# 1. title: App 1
# 2. title: App 2
# 3. title: App 3
```
### `developer_print_fields(dev_id, fields, count=100, lang='en', country='us')`
Prints multiple fields from all developer apps.
```python
scraper.developer_print_fields("5700313618786177705", ["title", "score"])
# Output:
# 1. title: App 1, score: 4.5
# 2. title: App 2, score: 4.2
```
### `developer_print_all(dev_id, count=100, lang='en', country='us')`
Prints all data for all developer apps as formatted JSON.
```python
scraper.developer_print_all("5700313618786177705")
# Output: Full JSON array with all apps
```
---
## Available Fields
- `appId` - App package name (e.g., "com.example.app")
- `title` - App name
- `description` - App description
- `icon` - App icon URL
- `url` - Play Store URL
- `developer` - Developer name
- `score` - Average rating (1-5)
- `scoreText` - Rating as text (e.g., "4.5")
- `currency` - Price currency (e.g., "USD")
- `price` - App price (0 if free)
- `free` - Boolean, true if free
---
## Practical Examples
### Analyze Developer Portfolio
```python
dev_id = "5700313618786177705"
apps = scraper.developer_analyze(dev_id)
print(f"Total apps: {len(apps)}")
print(f"Average rating: {sum(a['score'] for a in apps if a['score']) / len(apps):.2f}")
print(f"Free apps: {sum(1 for a in apps if a['free'])}")
print(f"Paid apps: {sum(1 for a in apps if not a['free'])}")
```
### Find Top-Rated Apps
```python
dev_id = "5700313618786177705"
apps = scraper.developer_get_fields(dev_id, ["title", "score"])
# Sort by rating
top_apps = sorted(apps, key=lambda x: x['score'] or 0, reverse=True)[:5]
for i, app in enumerate(top_apps, 1):
print(f"{i}. {app['title']}: {app['score']}★")
```
### Compare Free vs Paid Apps
```python
dev_id = "5700313618786177705"
apps = scraper.developer_get_fields(dev_id, ["title", "free", "price", "score"])
free_apps = [a for a in apps if a['free']]
paid_apps = [a for a in apps if not a['free']]
print(f"Free apps: {len(free_apps)} (avg rating: {sum(a['score'] or 0 for a in free_apps)/len(free_apps):.2f})")
print(f"Paid apps: {len(paid_apps)} (avg rating: {sum(a['score'] or 0 for a in paid_apps)/len(paid_apps):.2f})")
```
### Export Developer Apps
```python
import json
dev_id = "5700313618786177705"
apps = scraper.developer_analyze(dev_id)
with open('developer_apps.json', 'w') as f:
json.dump(apps, f, indent=2)
print(f"Exported {len(apps)} apps to developer_apps.json")
```
---
## Parameters
### Initialization
- `http_client` (str, optional) - HTTP client to use: "requests", "curl_cffi", "tls_client", "httpx", "urllib3", "cloudscraper", "aiohttp" (default: "requests")
### Method Parameters
- `dev_id` (str, required) - Developer ID (numeric or string)
- `count` (int, optional) - Maximum number of apps to return (default: 100)
- `lang` (str, optional) - Language code (default: 'en')
- `country` (str, optional) - Country code (default: 'us')
- `field` (str) - Single field name
- `fields` (List[str]) - List of field names
### Finding Developer IDs
**Method 1: From Developer Page URL**
- Numeric ID: `https://play.google.com/store/apps/dev?id=5700313618786177705`
- Developer ID: `5700313618786177705`
- String ID: `https://play.google.com/store/apps/developer?id=Google+LLC`
- Developer ID: `Google+LLC` or `Google LLC`
**Method 2: From App Page**
1. Go to any app by the developer
2. Click on the developer name
3. Extract ID from the URL
### Language & Country Codes
- **Language**: 'en', 'es', 'fr', 'de', 'ja', 'ko', 'pt', 'ru', 'zh', etc.
- **Country**: 'us', 'gb', 'ca', 'au', 'in', 'br', 'jp', 'kr', 'de', 'fr', etc.
---
## When to Use Each Method
- **`developer_analyze()`** - Need complete data for all apps
- **`developer_get_field()`** - Need just one field from all apps
- **`developer_get_fields()`** - Need specific fields from all apps (more efficient)
- **`developer_print_field()`** - Quick debugging/console output
- **`developer_print_fields()`** - Quick debugging of multiple fields
- **`developer_print_all()`** - Explore available data structure
---
## Advanced Features
### Rate Limiting
Built-in rate limiting (1 second delay between requests) prevents blocking.
### Error Handling
```python
from gplay_scraper import GPlayScraper, AppNotFoundError, NetworkError
scraper = GPlayScraper()
try:
apps = scraper.developer_analyze("invalid_dev_id")
except AppNotFoundError:
print("Developer not found")
except NetworkError:
print("Network error occurred")
```
### Multi-Region Data
```python
# Get developer apps from different regions
us_apps = scraper.developer_analyze("5700313618786177705", country="us")
uk_apps = scraper.developer_analyze("5700313618786177705", country="gb")
jp_apps = scraper.developer_analyze("5700313618786177705", country="jp", lang="ja")
```
### Pagination
```python
# Get first 50 apps
apps_batch1 = scraper.developer_analyze("5700313618786177705", count=50)
# Get more apps (library handles this automatically up to count limit)
apps_all = scraper.developer_analyze("5700313618786177705", count=200)
```
================================================
FILE: README/LIST_METHODS.md
================================================
# List Methods
Get top charts from Google Play Store (top free, top paid, top grossing).
## Quick Start
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Get top free apps
top_free = scraper.list_analyze("TOP_FREE", "GAME", count=50)
for app in top_free[:10]:
print(f"{app['title']}: {app['installs']} installs")
# Get specific fields
titles = scraper.list_get_field("TOP_FREE", "title", "APPLICATION")
print(titles)
# Get multiple fields
apps = scraper.list_get_fields("TOP_PAID", ["title", "price", "score"], "GAME")
print(apps)
```
---
## HTTP Clients
The library supports 7 HTTP clients with automatic fallback. If one fails, it tries the next.
### Supported Clients
1. **requests** (default) - Standard Python HTTP library
2. **curl_cffi** - cURL with browser impersonation
3. **tls_client** - Advanced TLS fingerprinting
4. **httpx** - Modern async-capable HTTP client
5. **urllib3** - Low-level HTTP client
6. **cloudscraper** - Cloudflare bypass
7. **aiohttp** - Async HTTP client
### Usage
```python
# Default (tries requests first, then others)
scraper = GPlayScraper()
# Specify a client
scraper = GPlayScraper(http_client="curl_cffi")
scraper = GPlayScraper(http_client="tls_client")
scraper = GPlayScraper(http_client="httpx")
```
### Installation
```bash
# Default
pip install requests
# Advanced clients (optional)
pip install curl-cffi
pip install tls-client
pip install httpx
pip install urllib3
pip install cloudscraper
pip install aiohttp
```
**Note:** The library automatically falls back to available clients if your preferred one fails.
---
## Methods
### `list_analyze(collection='TOP_FREE', category='APPLICATION', count=100, lang='en', country='us')`
Returns top chart apps as a list of dictionaries.
```python
apps = scraper.list_analyze("TOP_FREE", "GAME", count=50)
# Returns: [{'appId': '...', 'title': '...', 'installs': '...', ...}, ...]
```
### `list_get_field(collection, field, category='APPLICATION', count=100, lang='en', country='us')`
Returns a specific field from all chart apps.
```python
titles = scraper.list_get_field("TOP_FREE", "title", "APPLICATION")
# Returns: ['App 1', 'App 2', 'App 3', ...]
```
### `list_get_fields(collection, fields, category='APPLICATION', count=100, lang='en', country='us')`
Returns multiple fields from all chart apps.
```python
apps = scraper.list_get_fields("TOP_PAID", ["title", "price", "score"], "GAME")
# Returns: [{'title': 'App 1', 'price': 4.99, 'score': 4.5}, ...]
```
### `list_print_field(collection, field, category='APPLICATION', count=100, lang='en', country='us')`
Prints a specific field from all chart apps.
```python
scraper.list_print_field("TOP_FREE", "title", "APPLICATION", count=20)
# Output:
# 1. title: App 1
# 2. title: App 2
# 3. title: App 3
```
### `list_print_fields(collection, fields, category='APPLICATION', count=100, lang='en', country='us')`
Prints multiple fields from all chart apps.
```python
scraper.list_print_fields("TOP_FREE", ["title", "score"], "GAME", count=20)
# Output:
# 1. title: App 1, score: 4.5
# 2. title: App 2, score: 4.2
```
### `list_print_all(collection='TOP_FREE', category='APPLICATION', count=100, lang='en', country='us')`
Prints all data for all chart apps as formatted JSON.
```python
scraper.list_print_all("TOP_FREE", "GAME", count=50)
# Output: Full JSON array with all apps
```
---
## Available Fields
- `appId` - App package name (e.g., "com.example.app")
- `title` - App name
- `description` - App description
- `icon` - App icon URL
- `screenshots` - List of screenshot URLs
- `url` - Play Store URL
- `developer` - Developer name
- `genre` - App category
- `score` - Average rating (1-5)
- `scoreText` - Rating as text (e.g., "4.5")
- `installs` - Install count (e.g., "10,000,000+")
- `currency` - Price currency (e.g., "USD")
- `price` - App price (0 if free)
- `free` - Boolean, true if free
---
## Collection Types
### Available Collections
- **`TOP_FREE`** - Top free apps (most popular free apps)
- **`TOP_PAID`** - Top paid apps (most popular paid apps)
- **`TOP_GROSSING`** - Top grossing apps (highest revenue apps)
---
## Categories
### App Categories (36)
- `APPLICATION` - All apps (default)
- `ANDROID_WEAR` - Android Wear apps
- `ART_AND_DESIGN` - Art & design
- `AUTO_AND_VEHICLES` - Auto & vehicles
- `BEAUTY` - Beauty
- `BOOKS_AND_REFERENCE` - Books & reference
- `BUSINESS` - Business
- `COMICS` - Comics
- `COMMUNICATION` - Communication
- `DATING` - Dating
- `EDUCATION` - Education
- `ENTERTAINMENT` - Entertainment
- `EVENTS` - Events
- `FINANCE` - Finance
- `FOOD_AND_DRINK` - Food & drink
- `HEALTH_AND_FITNESS` - Health & fitness
- `HOUSE_AND_HOME` - House & home
- `LIBRARIES_AND_DEMO` - Libraries & demo
- `LIFESTYLE` - Lifestyle
- `MAPS_AND_NAVIGATION` - Maps & navigation
- `MEDICAL` - Medical
- `MUSIC_AND_AUDIO` - Music & audio
- `NEWS_AND_MAGAZINES` - News & magazines
- `PARENTING` - Parenting
- `PERSONALIZATION` - Personalization
- `PHOTOGRAPHY` - Photography
- `PRODUCTIVITY` - Productivity
- `SHOPPING` - Shopping
- `SOCIAL` - Social
- `SPORTS` - Sports
- `TOOLS` - Tools
- `TRAVEL_AND_LOCAL` - Travel & local
- `VIDEO_PLAYERS` - Video players & editors
- `WATCH_FACE` - Watch faces
- `WEATHER` - Weather
- `FAMILY` - Family
### Game Categories (18)
- `GAME` - All games
- `GAME_ACTION` - Action games
- `GAME_ADVENTURE` - Adventure games
- `GAME_ARCADE` - Arcade games
- `GAME_BOARD` - Board games
- `GAME_CARD` - Card games
- `GAME_CASINO` - Casino games
- `GAME_CASUAL` - Casual games
- `GAME_EDUCATIONAL` - Educational games
- `GAME_MUSIC` - Music games
- `GAME_PUZZLE` - Puzzle games
- `GAME_RACING` - Racing games
- `GAME_ROLE_PLAYING` - Role playing games
- `GAME_SIMULATION` - Simulation games
- `GAME_SPORTS` - Sports games
- `GAME_STRATEGY` - Strategy games
- `GAME_TRIVIA` - Trivia games
- `GAME_WORD` - Word games
---
## Practical Examples
### Top Free Games Analysis
```python
top_games = scraper.list_analyze("TOP_FREE", "GAME", count=100)
print(f"Total games: {len(top_games)}")
print(f"Average rating: {sum(a['score'] for a in top_games if a['score']) / len(top_games):.2f}")
print(f"\nTop 5 games:")
for i, game in enumerate(top_games[:5], 1):
print(f"{i}. {game['title']} - {game['score']}★ - {game['installs']} installs")
```
### Compare Free vs Paid Apps
```python
top_free = scraper.list_get_fields("TOP_FREE", ["title", "score", "installs"], "APPLICATION", count=50)
top_paid = scraper.list_get_fields("TOP_PAID", ["title", "score", "price"], "APPLICATION", count=50)
free_avg = sum(a['score'] or 0 for a in top_free) / len(top_free)
paid_avg = sum(a['score'] or 0 for a in top_paid) / len(top_paid)
print(f"Top Free Apps - Avg Rating: {free_avg:.2f}")
print(f"Top Paid Apps - Avg Rating: {paid_avg:.2f}")
```
### Find Highest Grossing Apps
```python
top_grossing = scraper.list_get_fields("TOP_GROSSING", ["title", "developer", "genre"], "APPLICATION", count=20)
print("Top 10 Highest Grossing Apps:")
for i, app in enumerate(top_grossing[:10], 1):
print(f"{i}. {app['title']} by {app['developer']} ({app['genre']})")
```
### Category Comparison
```python
categories = ["GAME", "SOCIAL", "PRODUCTIVITY", "ENTERTAINMENT"]
for category in categories:
apps = scraper.list_get_fields("TOP_FREE", ["title", "score"], category, count=10)
avg_score = sum(a['score'] or 0 for a in apps) / len(apps)
print(f"{category}: {avg_score:.2f}★ average")
```
### Game Genre Analysis
```python
game_genres = ["GAME_ACTION", "GAME_PUZZLE", "GAME_CASUAL", "GAME_STRATEGY"]
for genre in game_genres:
games = scraper.list_get_fields("TOP_FREE", ["title", "score", "installs"], genre, count=5)
print(f"\n{genre}:")
for i, game in enumerate(games, 1):
print(f" {i}. {game['title']} - {game['score']}★")
```
### Export Top Charts
```python
import json
top_free = scraper.list_analyze("TOP_FREE", "GAME", count=100)
with open('top_free_games.json', 'w') as f:
json.dump(top_free, f, indent=2)
print(f"Exported {len(top_free)} games to top_free_games.json")
```
### Track Chart Positions
```python
import time
import json
from datetime import datetime
def track_charts():
snapshot = {
"timestamp": datetime.now().isoformat(),
"top_free": scraper.list_get_fields("TOP_FREE", ["title", "score"], "GAME", count=10),
"top_paid": scraper.list_get_fields("TOP_PAID", ["title", "price"], "GAME", count=10)
}
with open(f'charts_{datetime.now().strftime("%Y%m%d")}.json', 'w') as f:
json.dump(snapshot, f, indent=2)
print(f"Snapshot saved at {snapshot['timestamp']}")
track_charts()
```
---
## Parameters
### Initialization
- `http_client` (str, optional) - HTTP client to use: "requests", "curl_cffi", "tls_client", "httpx", "urllib3", "cloudscraper", "aiohttp" (default: "requests")
### Method Parameters
- `collection` (str) - Chart type: "TOP_FREE", "TOP_PAID", "TOP_GROSSING" (default: "TOP_FREE")
- `category` (str, optional) - Category filter (default: "APPLICATION")
- `count` (int, optional) - Maximum number of apps to return (default: 100)
- `lang` (str, optional) - Language code (default: 'en')
- `country` (str, optional) - Country code (default: 'us')
- `field` (str) - Single field name
- `fields` (List[str]) - List of field names
### Language & Country Codes
- **Language**: 'en', 'es', 'fr', 'de', 'ja', 'ko', 'pt', 'ru', 'zh', etc.
- **Country**: 'us', 'gb', 'ca', 'au', 'in', 'br', 'jp', 'kr', 'de', 'fr', etc.
---
## When to Use Each Method
- **`list_analyze()`** - Need complete data for all chart apps
- **`list_get_field()`** - Need just one field from all apps
- **`list_get_fields()`** - Need specific fields from all apps (more efficient)
- **`list_print_field()`** - Quick debugging/console output
- **`list_print_fields()`** - Quick debugging of multiple fields
- **`list_print_all()`** - Explore available data structure
---
## Advanced Features
### Rate Limiting
Built-in rate limiting (1 second delay between requests) prevents blocking.
### Error Handling
```python
from gplay_scraper import GPlayScraper, AppNotFoundError, NetworkError
scraper = GPlayScraper()
try:
apps = scraper.list_analyze("INVALID_COLLECTION", "GAME")
except AppNotFoundError:
print("Collection not found")
except NetworkError:
print("Network error occurred")
```
### Multi-Region Charts
```python
# Get charts from different regions
us_charts = scraper.list_analyze("TOP_FREE", "GAME", country="us")
uk_charts = scraper.list_analyze("TOP_FREE", "GAME", country="gb")
jp_charts = scraper.list_analyze("TOP_FREE", "GAME", country="jp", lang="ja")
print(f"US Top Game: {us_charts[0]['title']}")
print(f"UK Top Game: {uk_charts[0]['title']}")
print(f"JP Top Game: {jp_charts[0]['title']}")
```
### Batch Analysis
```python
# Analyze multiple collections at once
collections = ["TOP_FREE", "TOP_PAID", "TOP_GROSSING"]
results = {}
for collection in collections:
apps = scraper.list_get_fields(collection, ["title", "score"], "GAME", count=10)
results[collection] = apps
print(f"{collection}: {len(apps)} apps retrieved")
```
================================================
FILE: README/README.md
================================================
# GPlay Scraper Documentation
Complete documentation for all 7 method types in GPlay Scraper.
## 📚 Method Documentation
### [App Methods](APP_METHODS.md)
Extract comprehensive app data with 65+ fields including ratings, installs, pricing, screenshots, permissions, and technical details.
**Key Features:**
- 65+ data fields per app
- Basic info, ratings, installs, pricing
- Media content (screenshots, videos, icons)
- Technical specs (version, size, Android version)
- Developer information and contact details
**Use Cases:** App analysis, competitive research, market intelligence, data collection
---
### [Search Methods](SEARCH_METHODS.md)
Search Google Play Store apps by keyword with filtering and pagination.
**Key Features:**
- Search by keyword, app name, or category
- Filter and paginate results
- Get app titles, developers, ratings, prices
- Multi-language and multi-region support
**Use Cases:** App discovery, market research, competitor analysis, trend tracking
---
### [Reviews Methods](REVIEWS_METHODS.md)
Extract user reviews with ratings, timestamps, and detailed feedback for sentiment analysis.
**Key Features:**
- Get reviews with ratings (1-5 stars)
- Review text, timestamps, app versions
- Reviewer names and helpful vote counts
- Sort by newest, relevant, or highest rated
**Use Cases:** Sentiment analysis, user feedback, app improvement, competitive monitoring
---
### [Developer Methods](DEVELOPER_METHODS.md)
Get all apps published by a specific developer using their developer ID.
**Key Features:**
- Complete app portfolio for any developer
- Track developer's app performance
- Analyze ratings and install counts
- Monitor developer's market presence
**Use Cases:** Developer research, portfolio analysis, competitive intelligence, market tracking
---
### [List Methods](LIST_METHODS.md)
Access Google Play Store top charts including top free, top paid, and top grossing apps by category.
**Key Features:**
- Top free, top paid, top grossing charts
- 54 categories (36 app + 18 game)
- Ranked lists with install counts and ratings
- Trending apps and market leaders
**Use Cases:** Market trends, category analysis, competitive benchmarking, app discovery
---
### [Similar Methods](SIMILAR_METHODS.md)
Find apps similar to a reference app for competitive analysis and market research.
**Key Features:**
- Discover competitor apps
- Find similar/related apps
- Get titles, developers, ratings, pricing
- Competitive analysis and positioning
**Use Cases:** Competitive analysis, market research, app discovery, positioning strategy
---
### [Suggest Methods](SUGGEST_METHODS.md)
Get search suggestions and autocomplete from Google Play Store for keyword discovery and ASO.
**Key Features:**
- Autocomplete suggestions
- Popular search terms
- Nested keyword discovery
- Multi-language support
**Use Cases:** Keyword research, ASO optimization, content strategy, market insights
---
## 🚀 Quick Start
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# App Methods
scraper.app_print_all("com.whatsapp")
# Search Methods
scraper.search_print_all("fitness tracker", count=20)
# Reviews Methods
scraper.reviews_print_all("com.whatsapp", count=100, sort="NEWEST")
# Developer Methods
scraper.developer_print_all("5700313618786177705", count=50)
# List Methods
scraper.list_print_all("TOP_FREE", "GAME", count=50)
# Similar Methods
scraper.similar_print_all("com.whatsapp", count=30)
# Suggest Methods
scraper.suggest_print_all("photo editor", count=10)
```
## 📖 Method Pattern
Each method type follows the same pattern with 6 functions:
- **`analyze()`** - Get all data as dictionary/list
- **`get_field()`** - Get single field value
- **`get_fields()`** - Get multiple fields as dictionary
- **`print_field()`** - Print single field to console
- **`print_fields()`** - Print multiple fields to console
- **`print_all()`** - Print all data as formatted JSON
## 🌍 Multi-Language & Multi-Region
All methods support multi-language and multi-region parameters:
```python
# Get data in Spanish from Spain
scraper.app_analyze("com.whatsapp", lang="es", country="es")
# Get data in Japanese from Japan
scraper.search_analyze("game", count=20, lang="ja", country="jp")
# Get data in French from France
scraper.reviews_analyze("com.whatsapp", count=50, lang="fr", country="fr")
```
**Supported:**
- **Languages:** 100+ (en, es, fr, de, ja, ko, zh, ar, pt, ru, etc.)
- **Countries:** 150+ (us, gb, ca, au, in, br, jp, kr, de, fr, etc.)
## 🔧 HTTP Clients
All methods support 7 HTTP clients with automatic fallback:
```python
# Default (requests)
scraper = GPlayScraper()
# Specify client
scraper = GPlayScraper(http_client="curl_cffi")
scraper = GPlayScraper(http_client="tls_client")
scraper = GPlayScraper(http_client="httpx")
```
**Available Clients:**
1. **requests** (default) - Standard Python HTTP library
2. **curl_cffi** - Browser impersonation with TLS fingerprinting
3. **tls_client** - Custom TLS fingerprinting
4. **httpx** - Modern async-capable HTTP client
5. **urllib3** - Low-level HTTP client
6. **cloudscraper** - Cloudflare bypass capabilities
7. **aiohttp** - Async HTTP client
## 📊 What Can You Scrape?
### App Data (65+ Fields)
- Basic: title, developer, description, category, genre
- Ratings: score, ratings count, histogram
- Installs: install count ranges, statistics
- Pricing: free/paid, price, in-app purchases
- Media: icon, screenshots, video, header image
- Technical: version, size, Android version, dates
- Content: age rating, privacy policy, contact info
- Features: permissions, what's new, website
### Search & Discovery
- Search apps by keyword
- Get search suggestions
- Find similar/competitor apps
- Access top charts by category
### Developer Intelligence
- Complete app portfolio
- Performance tracking
- Market presence analysis
### User Reviews
- Reviews with ratings and text
- Timestamps and app versions
- Reviewer names and votes
- Filter by sort options
### Market Research
- Multi-language support (100+ languages)
- Multi-region data (150+ countries)
- Localized pricing and availability
- Competitive analysis
## 🎯 Use Cases
**Market Research**
- Analyze competitor apps
- Track market trends
- Identify opportunities
- Benchmark performance
**App Development**
- Monitor user feedback
- Track app performance
- Analyze competitors
- Optimize app store presence
**Data Analysis**
- Collect app data for research
- Sentiment analysis from reviews
- Market intelligence reports
- Machine learning datasets
**Business Intelligence**
- Competitive monitoring
- Market positioning
- Trend analysis
- Strategic planning
## 📄 License
This project is licensed under the MIT License.
---
**For detailed documentation on each method type, click the links above.**
================================================
FILE: README/REVIEWS_METHODS.md
================================================
# Reviews Methods
Extract user reviews from Google Play Store apps with ratings, content, and metadata.
## Quick Start
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Get reviews
reviews = scraper.reviews_analyze("com.whatsapp", count=100, sort="NEWEST")
for review in reviews[:5]:
print(f"{review['userName']}: {review['score']}★")
print(f" {review['content'][:100]}...")
# Get specific fields
scores = scraper.reviews_get_field("com.whatsapp", "score", count=100)
print(f"Average: {sum(scores)/len(scores):.2f}★")
# Get multiple fields
reviews = scraper.reviews_get_fields("com.whatsapp", ["userName", "score", "content"], count=50)
print(reviews)
```
---
## HTTP Clients
The library supports 7 HTTP clients with automatic fallback. If one fails, it tries the next.
### Supported Clients
1. **requests** (default) - Standard Python HTTP library
2. **curl_cffi** - cURL with browser impersonation
3. **tls_client** - Advanced TLS fingerprinting
4. **httpx** - Modern async-capable HTTP client
5. **urllib3** - Low-level HTTP client
6. **cloudscraper** - Cloudflare bypass
7. **aiohttp** - Async HTTP client
### Usage
```python
# Default (tries requests first, then others)
scraper = GPlayScraper()
# Specify a client
scraper = GPlayScraper(http_client="curl_cffi")
scraper = GPlayScraper(http_client="tls_client")
scraper = GPlayScraper(http_client="httpx")
```
### Installation
```bash
# Default
pip install requests
# Advanced clients (optional)
pip install curl-cffi
pip install tls-client
pip install httpx
pip install urllib3
pip install cloudscraper
pip install aiohttp
```
**Note:** The library automatically falls back to available clients if your preferred one fails.
---
## Methods
### `reviews_analyze(app_id, count=100, lang='en', country='us', sort='NEWEST')`
Returns reviews as a list of dictionaries.
```python
reviews = scraper.reviews_analyze("com.whatsapp", count=100, sort="NEWEST")
# Returns: [{'reviewId': '...', 'userName': '...', 'score': 5, 'content': '...', ...}, ...]
```
### `reviews_get_field(app_id, field, count=100, lang='en', country='us', sort='NEWEST')`
Returns a specific field from all reviews.
```python
scores = scraper.reviews_get_field("com.whatsapp", "score", count=100)
# Returns: [5, 4, 5, 3, 4, ...]
```
### `reviews_get_fields(app_id, fields, count=100, lang='en', country='us', sort='NEWEST')`
Returns multiple fields from all reviews.
```python
reviews = scraper.reviews_get_fields("com.whatsapp", ["userName", "score", "content"], count=50)
# Returns: [{'userName': 'John', 'score': 5, 'content': 'Great app!'}, ...]
```
### `reviews_print_field(app_id, field, count=100, lang='en', country='us', sort='NEWEST')`
Prints a specific field from all reviews.
```python
scraper.reviews_print_field("com.whatsapp", "content", count=20)
# Output:
# 1. content: Great app!
# 2. content: Love it
# 3. content: Needs improvement
```
### `reviews_print_fields(app_id, fields, count=100, lang='en', country='us', sort='NEWEST')`
Prints multiple fields from all reviews.
```python
scraper.reviews_print_fields("com.whatsapp", ["userName", "score"], count=20)
# Output:
# userName: John, score: 5
# userName: Jane, score: 4
```
### `reviews_print_all(app_id, count=100, lang='en', country='us', sort='NEWEST')`
Prints all review data as formatted JSON.
```python
scraper.reviews_print_all("com.whatsapp", count=50)
# Output: Full JSON array with all reviews
```
---
## Available Fields
- `reviewId` - Unique review ID
- `userName` - Reviewer name
- `userImage` - Reviewer avatar URL
- `score` - Review rating (1-5 stars)
- `content` - Review text/comment
- `thumbsUpCount` - Number of helpful votes
- `appVersion` - App version reviewed
- `at` - Review timestamp (ISO 8601 format)
---
## Sort Options
- **`NEWEST`** (default) - Most recent reviews first
- **`RELEVANT`** - Most relevant/helpful reviews
- **`RATING`** - Sorted by rating (highest/lowest)
---
## Practical Examples
### Sentiment Analysis
```python
reviews = scraper.reviews_get_fields("com.whatsapp", ["score", "content"], count=200)
# Rating distribution
rating_dist = {1: 0, 2: 0, 3: 0, 4: 0, 5: 0}
for review in reviews:
rating_dist[review['score']] += 1
print("Rating Distribution:")
for rating, count in rating_dist.items():
print(f"{rating}★: {'█' * count} ({count})")
# Average rating
avg = sum(r['score'] for r in reviews) / len(reviews)
print(f"\nAverage: {avg:.2f}★")
```
### Find Common Issues
```python
reviews = scraper.reviews_get_fields("com.whatsapp", ["score", "content"], count=100, sort="RATING")
# Get low-rated reviews
low_rated = [r for r in reviews if r['score'] <= 2]
print(f"Found {len(low_rated)} low-rated reviews:")
for review in low_rated[:10]:
print(f"- {review['content'][:100]}...")
```
### Track Review Trends
```python
from datetime import datetime
reviews = scraper.reviews_get_fields("com.whatsapp", ["at", "score"], count=500, sort="NEWEST")
# Group by month
monthly_scores = {}
for review in reviews:
date = datetime.fromisoformat(review['at'])
month_key = date.strftime("%Y-%m")
if month_key not in monthly_scores:
monthly_scores[month_key] = []
monthly_scores[month_key].append(review['score'])
# Calculate monthly averages
for month, scores in sorted(monthly_scores.items()):
avg = sum(scores) / len(scores)
print(f"{month}: {avg:.2f}★ ({len(scores)} reviews)")
```
### Compare App Versions
```python
reviews = scraper.reviews_get_fields("com.whatsapp", ["appVersion", "score"], count=300)
# Group by version
version_scores = {}
for review in reviews:
version = review['appVersion'] or "Unknown"
if version not in version_scores:
version_scores[version] = []
version_scores[version].append(review['score'])
# Show version ratings
for version, scores in sorted(version_scores.items()):
if len(scores) >= 5: # Only versions with 5+ reviews
avg = sum(scores) / len(scores)
print(f"v{version}: {avg:.2f}★ ({len(scores)} reviews)")
```
### Export Reviews to CSV
```python
import csv
reviews = scraper.reviews_analyze("com.whatsapp", count=500)
with open('reviews.csv', 'w', newline='', encoding='utf-8') as f:
writer = csv.DictWriter(f, fieldnames=['userName', 'score', 'content', 'at', 'appVersion'])
writer.writeheader()
for review in reviews:
writer.writerow({
'userName': review['userName'],
'score': review['score'],
'content': review['content'],
'at': review['at'],
'appVersion': review['appVersion']
})
print(f"Exported {len(reviews)} reviews to reviews.csv")
```
### Identify Top Reviewers
```python
reviews = scraper.reviews_get_fields("com.whatsapp", ["userName", "thumbsUpCount"], count=200)
# Sort by helpful votes
top_reviewers = sorted(reviews, key=lambda x: x['thumbsUpCount'] or 0, reverse=True)[:10]
print("Top 10 Most Helpful Reviewers:")
for i, review in enumerate(top_reviewers, 1):
print(f"{i}. {review['userName']}: {review['thumbsUpCount']} helpful votes")
```
### Monitor Recent Feedback
```python
import time
from datetime import datetime
def monitor_reviews(app_id, interval=3600):
"""Check for new reviews every hour"""
last_check = datetime.now()
while True:
reviews = scraper.reviews_get_fields(app_id, ["at", "score", "content"], count=50, sort="NEWEST")
new_reviews = [r for r in reviews if datetime.fromisoformat(r['at']) > last_check]
if new_reviews:
print(f"\n{len(new_reviews)} new reviews:")
for review in new_reviews:
print(f"- {review['score']}★: {review['content'][:80]}...")
last_check = datetime.now()
time.sleep(interval)
# Run monitor (Ctrl+C to stop)
# monitor_reviews("com.whatsapp")
```
### Keyword Analysis
```python
from collections import Counter
import re
reviews = scraper.reviews_get_field("com.whatsapp", "content", count=500)
# Extract words
words = []
for content in reviews:
if content:
words.extend(re.findall(r'\b\w+\b', content.lower()))
# Remove common words
stop_words = {'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are', 'was', 'were', 'in', 'on', 'at', 'to', 'for'}
filtered_words = [w for w in words if w not in stop_words and len(w) > 3]
# Top keywords
top_keywords = Counter(filtered_words).most_common(20)
print("Top Keywords in Reviews:")
for word, count in top_keywords:
print(f"{word}: {count}")
```
---
## Parameters
### Initialization
- `http_client` (str, optional) - HTTP client to use: "requests", "curl_cffi", "tls_client", "httpx", "urllib3", "cloudscraper", "aiohttp" (default: "requests")
### Method Parameters
- `app_id` (str, required) - App package name
- `count` (int, optional) - Maximum number of reviews to return (default: 100)
- `lang` (str, optional) - Language code (default: 'en')
- `country` (str, optional) - Country code (default: 'us')
- `sort` (str, optional) - Sort order: "NEWEST", "RELEVANT", "RATING" (default: "NEWEST")
- `field` (str) - Single field name
- `fields` (List[str]) - List of field names
### Language & Country Codes
- **Language**: 'en', 'es', 'fr', 'de', 'ja', 'ko', 'pt', 'ru', 'zh', etc.
- **Country**: 'us', 'gb', 'ca', 'au', 'in', 'br', 'jp', 'kr', 'de', 'fr', etc.
---
## When to Use Each Method
- **`reviews_analyze()`** - Need complete review data for analysis
- **`reviews_get_field()`** - Need just one field (e.g., all scores)
- **`reviews_get_fields()`** - Need specific fields (more efficient)
- **`reviews_print_field()`** - Quick debugging/console output
- **`reviews_print_fields()`** - Quick debugging of multiple fields
- **`reviews_print_all()`** - Explore available data structure
---
## Advanced Features
### Rate Limiting
Built-in rate limiting (1 second delay between requests) prevents blocking.
### Batch Fetching
Reviews are fetched in batches of 50. The library automatically handles pagination.
```python
# Fetch 500 reviews (10 batches of 50)
reviews = scraper.reviews_analyze("com.whatsapp", count=500)
print(f"Fetched {len(reviews)} reviews")
```
### Error Handling
```python
from gplay_scraper import GPlayScraper, AppNotFoundError, NetworkError
scraper = GPlayScraper()
try:
reviews = scraper.reviews_analyze("invalid.app.id")
except AppNotFoundError:
print("App not found")
except NetworkError:
print("Network error occurred")
```
### Multi-Region Reviews
```python
# Get reviews from different regions
us_reviews = scraper.reviews_analyze("com.whatsapp", country="us", count=100)
uk_reviews = scraper.reviews_analyze("com.whatsapp", country="gb", count=100)
jp_reviews = scraper.reviews_analyze("com.whatsapp", country="jp", lang="ja", count=100)
print(f"US avg: {sum(r['score'] for r in us_reviews)/len(us_reviews):.2f}★")
print(f"UK avg: {sum(r['score'] for r in uk_reviews)/len(uk_reviews):.2f}★")
print(f"JP avg: {sum(r['score'] for r in jp_reviews)/len(jp_reviews):.2f}★")
```
### Sort Comparison
```python
# Compare different sort orders
newest = scraper.reviews_get_fields("com.whatsapp", ["score"], count=100, sort="NEWEST")
relevant = scraper.reviews_get_fields("com.whatsapp", ["score"], count=100, sort="RELEVANT")
rating = scraper.reviews_get_fields("com.whatsapp", ["score"], count=100, sort="RATING")
print(f"Newest avg: {sum(r['score'] for r in newest)/len(newest):.2f}★")
print(f"Relevant avg: {sum(r['score'] for r in relevant)/len(relevant):.2f}★")
print(f"Rating avg: {sum(r['score'] for r in rating)/len(rating):.2f}★")
```
================================================
FILE: README/SEARCH_METHODS.md
================================================
# Search Methods
Search for apps on Google Play Store by keyword, app name, or category.
## Quick Start
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Search for apps
results = scraper.search_analyze("social media", count=20)
for app in results:
print(f"{app['title']}: {app['score']}★ by {app['developer']}")
# Get specific fields
titles = scraper.search_get_field("fitness tracker", "title")
print(titles)
# Get multiple fields
apps = scraper.search_get_fields("photo editor", ["title", "score", "free"])
print(apps)
```
---
## HTTP Clients
The library supports 7 HTTP clients with automatic fallback. If one fails, it tries the next.
### Supported Clients
1. **requests** (default) - Standard Python HTTP library
2. **curl_cffi** - cURL with browser impersonation
3. **tls_client** - Advanced TLS fingerprinting
4. **httpx** - Modern async-capable HTTP client
5. **urllib3** - Low-level HTTP client
6. **cloudscraper** - Cloudflare bypass
7. **aiohttp** - Async HTTP client
### Usage
```python
# Default (tries requests first, then others)
scraper = GPlayScraper()
# Specify a client
scraper = GPlayScraper(http_client="curl_cffi")
scraper = GPlayScraper(http_client="tls_client")
scraper = GPlayScraper(http_client="httpx")
```
### Installation
```bash
# Default
pip install requests
# Advanced clients (optional)
pip install curl-cffi
pip install tls-client
pip install httpx
pip install urllib3
pip install cloudscraper
pip install aiohttp
```
**Note:** The library automatically falls back to available clients if your preferred one fails.
---
## Methods
### `search_analyze(query, count=100, lang='en', country='us')`
Returns search results as a list of dictionaries.
```python
results = scraper.search_analyze("social media", count=20)
# Returns: [{'appId': '...', 'title': '...', 'score': 4.5, ...}, ...]
```
### `search_get_field(query, field, count=100, lang='en', country='us')`
Returns a specific field from all search results.
```python
titles = scraper.search_get_field("fitness tracker", "title")
# Returns: ['App 1', 'App 2', 'App 3', ...]
```
### `search_get_fields(query, fields, count=100, lang='en', country='us')`
Returns multiple fields from all search results.
```python
apps = scraper.search_get_fields("photo editor", ["title", "score", "free"])
# Returns: [{'title': 'App 1', 'score': 4.5, 'free': True}, ...]
```
### `search_print_field(query, field, count=100, lang='en', country='us')`
Prints a specific field from all search results.
```python
scraper.search_print_field("social media", "title", count=10)
# Output:
# 0. title: App 1
# 1. title: App 2
# 2. title: App 3
```
### `search_print_fields(query, fields, count=100, lang='en', country='us')`
Prints multiple fields from all search results.
```python
scraper.search_print_fields("social media", ["title", "score"], count=10)
# Output:
# 0. title: App 1, score: 4.5
# 1. title: App 2, score: 4.2
```
### `search_print_all(query, count=100, lang='en', country='us')`
Prints all data for all search results as formatted JSON.
```python
scraper.search_print_all("social media", count=20)
# Output: Full JSON array with all search results
```
---
## Available Fields
- `appId` - App package name (e.g., "com.example.app")
- `title` - App name
- `description` - App description/summary
- `icon` - App icon URL
- `url` - Play Store URL
- `developer` - Developer name
- `score` - Average rating (1-5)
- `scoreText` - Rating as text (e.g., "4.5")
- `currency` - Price currency (e.g., "USD")
- `price` - App price (0 if free)
- `free` - Boolean, true if free
---
## Practical Examples
### Find Top-Rated Apps
```python
results = scraper.search_get_fields("productivity", ["title", "score", "developer"], count=50)
# Filter high-rated apps
top_rated = [app for app in results if app['score'] and app['score'] >= 4.5]
top_rated.sort(key=lambda x: x['score'], reverse=True)
print("Top-Rated Productivity Apps:")
for i, app in enumerate(top_rated[:10], 1):
print(f"{i}. {app['title']}: {app['score']}★ by {app['developer']}")
```
### Compare Free vs Paid Apps
```python
results = scraper.search_get_fields("photo editor", ["title", "free", "price", "score"], count=50)
free_apps = [app for app in results if app['free']]
paid_apps = [app for app in results if not app['free']]
free_avg = sum(app['score'] or 0 for app in free_apps) / len(free_apps) if free_apps else 0
paid_avg = sum(app['score'] or 0 for app in paid_apps) / len(paid_apps) if paid_apps else 0
print(f"Free apps: {len(free_apps)} (avg: {free_avg:.2f}★)")
print(f"Paid apps: {len(paid_apps)} (avg: {paid_avg:.2f}★)")
```
### Market Research
```python
keywords = ["fitness", "meditation", "diet", "sleep tracker"]
for keyword in keywords:
results = scraper.search_get_fields(keyword, ["title", "score"], count=10)
avg_score = sum(app['score'] or 0 for app in results) / len(results)
print(f"{keyword}: {len(results)} apps, avg {avg_score:.2f}★")
```
### Find Competitors
```python
query = "task manager"
results = scraper.search_get_fields(query, ["title", "developer", "score", "free"], count=30)
print(f"Competitors for '{query}':")
for i, app in enumerate(results[:15], 1):
price = "Free" if app['free'] else f"${app.get('price', 'N/A')}"
print(f"{i}. {app['title']} by {app['developer']} - {app['score']}★ ({price})")
```
### Export Search Results
```python
import json
query = "language learning"
results = scraper.search_analyze(query, count=100)
with open(f'search_{query.replace(" ", "_")}.json', 'w') as f:
json.dump(results, f, indent=2)
print(f"Exported {len(results)} results for '{query}'")
```
### Multi-Keyword Search
```python
keywords = ["vpn", "proxy", "security"]
all_results = {}
for keyword in keywords:
results = scraper.search_get_fields(keyword, ["appId", "title", "score"], count=20)
all_results[keyword] = results
print(f"{keyword}: {len(results)} apps found")
# Find apps appearing in multiple searches
app_ids = {}
for keyword, results in all_results.items():
for app in results:
app_id = app['appId']
if app_id not in app_ids:
app_ids[app_id] = {'title': app['title'], 'keywords': []}
app_ids[app_id]['keywords'].append(keyword)
# Apps in multiple categories
multi_category = {aid: data for aid, data in app_ids.items() if len(data['keywords']) > 1}
print(f"\nApps in multiple categories: {len(multi_category)}")
for app_id, data in list(multi_category.items())[:5]:
print(f"- {data['title']}: {', '.join(data['keywords'])}")
```
### Analyze Developer Presence
```python
from collections import Counter
query = "puzzle game"
results = scraper.search_get_field(query, "developer", count=100)
# Count apps per developer
developer_counts = Counter(results)
top_developers = developer_counts.most_common(10)
print(f"Top Developers in '{query}':")
for developer, count in top_developers:
print(f"{developer}: {count} apps")
```
### Price Range Analysis
```python
query = "premium photo editor"
results = scraper.search_get_fields(query, ["title", "price", "free"], count=50)
paid_apps = [app for app in results if not app['free'] and app['price']]
if paid_apps:
prices = [app['price'] for app in paid_apps]
print(f"Price Analysis for '{query}':")
print(f" Min: ${min(prices):.2f}")
print(f" Max: ${max(prices):.2f}")
print(f" Avg: ${sum(prices)/len(prices):.2f}")
print(f" Total paid apps: {len(paid_apps)}")
```
---
## Parameters
### Initialization
- `http_client` (str, optional) - HTTP client to use: "requests", "curl_cffi", "tls_client", "httpx", "urllib3", "cloudscraper", "aiohttp" (default: "requests")
### Method Parameters
- `query` (str, required) - Search keyword or phrase
- `count` (int, optional) - Maximum number of results to return (default: 100)
- `lang` (str, optional) - Language code (default: 'en')
- `country` (str, optional) - Country code (default: 'us')
- `field` (str) - Single field name
- `fields` (List[str]) - List of field names
### Search Query Tips
- Use specific keywords: "fitness tracker" vs "fitness"
- Try app categories: "puzzle game", "photo editor"
- Search by functionality: "vpn", "password manager"
- Use brand names: "google", "microsoft"
- Combine terms: "free music player"
### Language & Country Codes
- **Language**: 'en', 'es', 'fr', 'de', 'ja', 'ko', 'pt', 'ru', 'zh', etc.
- **Country**: 'us', 'gb', 'ca', 'au', 'in', 'br', 'jp', 'kr', 'de', 'fr', etc.
---
## When to Use Each Method
- **`search_analyze()`** - Need complete data for all search results
- **`search_get_field()`** - Need just one field from all results
- **`search_get_fields()`** - Need specific fields from all results (more efficient)
- **`search_print_field()`** - Quick debugging/console output
- **`search_print_fields()`** - Quick debugging of multiple fields
- **`search_print_all()`** - Explore available data structure
---
## Advanced Features
### Rate Limiting
Built-in rate limiting (1 second delay between requests) prevents blocking.
### Error Handling
```python
from gplay_scraper import GPlayScraper, AppNotFoundError, NetworkError
scraper = GPlayScraper()
try:
results = scraper.search_analyze("")
except ValueError:
print("Query cannot be empty")
except NetworkError:
print("Network error occurred")
```
### Multi-Region Search
```python
# Search in different regions
us_results = scraper.search_analyze("vpn", country="us", count=20)
uk_results = scraper.search_analyze("vpn", country="gb", count=20)
jp_results = scraper.search_analyze("vpn", country="jp", lang="ja", count=20)
print(f"US: {len(us_results)} results")
print(f"UK: {len(uk_results)} results")
print(f"JP: {len(jp_results)} results")
```
### Pagination
```python
# Get more results
results_20 = scraper.search_analyze("game", count=20)
results_50 = scraper.search_analyze("game", count=50)
results_100 = scraper.search_analyze("game", count=100)
print(f"20 results: {len(results_20)}")
print(f"50 results: {len(results_50)}")
print(f"100 results: {len(results_100)}")
```
### Search Result Filtering
```python
results = scraper.search_analyze("music player", count=50)
# Filter by rating
high_rated = [app for app in results if app['score'] and app['score'] >= 4.0]
# Filter by price
free_apps = [app for app in results if app['free']]
# Filter by developer
google_apps = [app for app in results if 'google' in app['developer'].lower()]
print(f"High rated: {len(high_rated)}")
print(f"Free: {len(free_apps)}")
print(f"Google: {len(google_apps)}")
```
================================================
FILE: README/SIMILAR_METHODS.md
================================================
# Similar Methods
Find similar and related apps on Google Play Store based on a reference app.
## Quick Start
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Get similar apps
similar = scraper.similar_analyze("com.whatsapp", count=20)
for app in similar:
print(f"{app['title']}: {app['score']}★ by {app['developer']}")
# Get specific fields
titles = scraper.similar_get_field("com.whatsapp", "title")
print(titles)
# Get multiple fields
apps = scraper.similar_get_fields("com.whatsapp", ["title", "score", "free"])
print(apps)
```
---
## HTTP Clients
The library supports 7 HTTP clients with automatic fallback. If one fails, it tries the next.
### Supported Clients
1. **requests** (default) - Standard Python HTTP library
2. **curl_cffi** - cURL with browser impersonation
3. **tls_client** - Advanced TLS fingerprinting
4. **httpx** - Modern async-capable HTTP client
5. **urllib3** - Low-level HTTP client
6. **cloudscraper** - Cloudflare bypass
7. **aiohttp** - Async HTTP client
### Usage
```python
# Default (tries requests first, then others)
scraper = GPlayScraper()
# Specify a client
scraper = GPlayScraper(http_client="curl_cffi")
scraper = GPlayScraper(http_client="tls_client")
scraper = GPlayScraper(http_client="httpx")
```
### Installation
```bash
# Default
pip install requests
# Advanced clients (optional)
pip install curl-cffi
pip install tls-client
pip install httpx
pip install urllib3
pip install cloudscraper
pip install aiohttp
```
**Note:** The library automatically falls back to available clients if your preferred one fails.
---
## Methods
### `similar_analyze(app_id, count=100, lang='en', country='us')`
Returns similar apps as a list of dictionaries.
```python
similar = scraper.similar_analyze("com.whatsapp", count=20)
# Returns: [{'appId': '...', 'title': '...', 'score': 4.5, ...}, ...]
```
### `similar_get_field(app_id, field, count=100, lang='en', country='us')`
Returns a specific field from all similar apps.
```python
titles = scraper.similar_get_field("com.whatsapp", "title")
# Returns: ['App 1', 'App 2', 'App 3', ...]
```
### `similar_get_fields(app_id, fields, count=100, lang='en', country='us')`
Returns multiple fields from all similar apps.
```python
apps = scraper.similar_get_fields("com.whatsapp", ["title", "score", "free"])
# Returns: [{'title': 'App 1', 'score': 4.5, 'free': True}, ...]
```
### `similar_print_field(app_id, field, count=100, lang='en', country='us')`
Prints a specific field from all similar apps.
```python
scraper.similar_print_field("com.whatsapp", "title", count=10)
# Output:
# 1. title: App 1
# 2. title: App 2
# 3. title: App 3
```
### `similar_print_fields(app_id, fields, count=100, lang='en', country='us')`
Prints multiple fields from all similar apps.
```python
scraper.similar_print_fields("com.whatsapp", ["title", "score"], count=10)
# Output:
# 1. title: App 1, score: 4.5
# 2. title: App 2, score: 4.2
```
### `similar_print_all(app_id, count=100, lang='en', country='us')`
Prints all data for all similar apps as formatted JSON.
```python
scraper.similar_print_all("com.whatsapp", count=20)
# Output: Full JSON array with all similar apps
```
---
## Available Fields
- `appId` - App package name (e.g., "com.example.app")
- `title` - App name
- `description` - App description
- `icon` - App icon URL
- `url` - Play Store URL
- `developer` - Developer name
- `score` - Average rating (1-5)
- `scoreText` - Rating as text (e.g., "4.5")
- `currency` - Price currency (e.g., "USD")
- `price` - App price (0 if free)
- `free` - Boolean, true if free
---
## Practical Examples
### Competitive Analysis
```python
app_id = "com.whatsapp"
similar = scraper.similar_get_fields(app_id, ["title", "score", "developer"], count=30)
print(f"Competitors of {app_id}:")
for i, app in enumerate(similar[:10], 1):
print(f"{i}. {app['title']}: {app['score']}★ by {app['developer']}")
# Calculate average competitor rating
avg_score = sum(app['score'] or 0 for app in similar) / len(similar)
print(f"\nAverage competitor rating: {avg_score:.2f}★")
```
### Find Better Alternatives
```python
app_id = "com.example.app"
my_app = scraper.app_get_field(app_id, "score")
similar = scraper.similar_get_fields(app_id, ["title", "score", "url"], count=50)
# Find apps with higher ratings
better_apps = [app for app in similar if app['score'] and app['score'] > my_app]
better_apps.sort(key=lambda x: x['score'], reverse=True)
print(f"Apps better than {app_id} ({my_app}★):")
for app in better_apps[:10]:
print(f"- {app['title']}: {app['score']}★")
```
### Market Positioning
```python
app_id = "com.whatsapp"
similar = scraper.similar_get_fields(app_id, ["title", "free", "price", "score"], count=50)
free_apps = [app for app in similar if app['free']]
paid_apps = [app for app in similar if not app['free']]
print(f"Market Analysis for {app_id}:")
print(f" Free competitors: {len(free_apps)}")
print(f" Paid competitors: {len(paid_apps)}")
if free_apps:
print(f" Free avg rating: {sum(a['score'] or 0 for a in free_apps)/len(free_apps):.2f}★")
if paid_apps:
print(f" Paid avg rating: {sum(a['score'] or 0 for a in paid_apps)/len(paid_apps):.2f}★")
```
### Developer Overlap Analysis
```python
from collections import Counter
app_id = "com.whatsapp"
similar = scraper.similar_get_field(app_id, "developer", count=50)
# Count apps per developer
developer_counts = Counter(similar)
top_developers = developer_counts.most_common(5)
print(f"Top developers in similar apps to {app_id}:")
for developer, count in top_developers:
print(f"{developer}: {count} apps")
```
### Export Similar Apps
```python
import json
app_id = "com.whatsapp"
similar = scraper.similar_analyze(app_id, count=50)
with open(f'similar_to_{app_id}.json', 'w') as f:
json.dump(similar, f, indent=2)
print(f"Exported {len(similar)} similar apps to similar_to_{app_id}.json")
```
### Compare Multiple Apps
```python
apps_to_compare = ["com.whatsapp", "com.telegram", "com.viber"]
all_similar = {}
for app_id in apps_to_compare:
similar = scraper.similar_get_fields(app_id, ["appId", "title"], count=20)
all_similar[app_id] = [app['appId'] for app in similar]
print(f"{app_id}: {len(similar)} similar apps")
# Find common competitors
common = set(all_similar[apps_to_compare[0]])
for app_id in apps_to_compare[1:]:
common &= set(all_similar[app_id])
print(f"\nCommon competitors: {len(common)}")
for app_id in list(common)[:5]:
title = scraper.app_get_field(app_id, "title")
print(f"- {title}")
```
### Feature Gap Analysis
```python
app_id = "com.whatsapp"
similar = scraper.similar_get_fields(app_id, ["title", "score"], count=30)
# Get top-rated competitors
top_competitors = sorted(similar, key=lambda x: x['score'] or 0, reverse=True)[:5]
print(f"Top-rated competitors of {app_id}:")
for i, app in enumerate(top_competitors, 1):
print(f"{i}. {app['title']}: {app['score']}★")
# You can then analyze these apps individually for features
```
### Price Comparison
```python
app_id = "com.example.paidapp"
my_price = scraper.app_get_field(app_id, "price")
similar = scraper.similar_get_fields(app_id, ["title", "price", "free"], count=50)
paid_similar = [app for app in similar if not app['free'] and app['price']]
if paid_similar:
prices = [app['price'] for app in paid_similar]
print(f"Price Comparison:")
print(f" Your app: ${my_price:.2f}")
print(f" Competitor min: ${min(prices):.2f}")
print(f" Competitor max: ${max(prices):.2f}")
print(f" Competitor avg: ${sum(prices)/len(prices):.2f}")
```
---
## Parameters
### Initialization
- `http_client` (str, optional) - HTTP client to use: "requests", "curl_cffi", "tls_client", "httpx", "urllib3", "cloudscraper", "aiohttp" (default: "requests")
### Method Parameters
- `app_id` (str, required) - App package name to find similar apps for
- `count` (int, optional) - Maximum number of similar apps to return (default: 100)
- `lang` (str, optional) - Language code (default: 'en')
- `country` (str, optional) - Country code (default: 'us')
- `field` (str) - Single field name
- `fields` (List[str]) - List of field names
### Language & Country Codes
- **Language**: 'en', 'es', 'fr', 'de', 'ja', 'ko', 'pt', 'ru', 'zh', etc.
- **Country**: 'us', 'gb', 'ca', 'au', 'in', 'br', 'jp', 'kr', 'de', 'fr', etc.
---
## When to Use Each Method
- **`similar_analyze()`** - Need complete data for all similar apps
- **`similar_get_field()`** - Need just one field from all similar apps
- **`similar_get_fields()`** - Need specific fields from all similar apps (more efficient)
- **`similar_print_field()`** - Quick debugging/console output
- **`similar_print_fields()`** - Quick debugging of multiple fields
- **`similar_print_all()`** - Explore available data structure
---
## Use Cases
### Competitive Intelligence
- Identify direct competitors
- Monitor competitor ratings and pricing
- Track market positioning
- Discover new entrants in your category
### Market Research
- Understand market landscape
- Analyze pricing strategies
- Identify market gaps
- Study successful competitors
### Product Development
- Find feature inspiration
- Identify differentiation opportunities
- Benchmark against competitors
- Discover user expectations
### Marketing Strategy
- Identify target audience overlap
- Study competitor positioning
- Find partnership opportunities
- Analyze market trends
---
## Advanced Features
### Rate Limiting
Built-in rate limiting (1 second delay between requests) prevents blocking.
### Error Handling
```python
from gplay_scraper import GPlayScraper, AppNotFoundError, NetworkError
scraper = GPlayScraper()
try:
similar = scraper.similar_analyze("invalid.app.id")
except AppNotFoundError:
print("App not found or no similar apps available")
except NetworkError:
print("Network error occurred")
```
### Multi-Region Similar Apps
```python
# Get similar apps from different regions
us_similar = scraper.similar_analyze("com.whatsapp", country="us", count=20)
uk_similar = scraper.similar_analyze("com.whatsapp", country="gb", count=20)
jp_similar = scraper.similar_analyze("com.whatsapp", country="jp", lang="ja", count=20)
print(f"US similar apps: {len(us_similar)}")
print(f"UK similar apps: {len(uk_similar)}")
print(f"JP similar apps: {len(jp_similar)}")
```
### Filtering Results
```python
similar = scraper.similar_analyze("com.whatsapp", count=50)
# Filter by rating
high_rated = [app for app in similar if app['score'] and app['score'] >= 4.0]
# Filter by price
free_apps = [app for app in similar if app['free']]
# Filter by developer
exclude_dev = [app for app in similar if app['developer'] != "WhatsApp LLC"]
print(f"High rated: {len(high_rated)}")
print(f"Free: {len(free_apps)}")
print(f"Other developers: {len(exclude_dev)}")
```
### Batch Analysis
```python
# Analyze similar apps for multiple apps
apps = ["com.whatsapp", "com.telegram", "com.viber"]
results = {}
for app_id in apps:
similar = scraper.similar_get_fields(app_id, ["title", "score"], count=10)
results[app_id] = similar
print(f"{app_id}: {len(similar)} similar apps found")
```
================================================
FILE: README/SUGGEST_METHODS.md
================================================
# Suggest Methods
Get search suggestions and autocomplete from Google Play Store for keyword discovery and ASO.
## Quick Start
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Get suggestions
suggestions = scraper.suggest_analyze("video", count=5)
print(suggestions)
# ['video player', 'video editor', 'video downloader', 'video maker', 'video call']
# Get nested suggestions
nested = scraper.suggest_nested("video", count=3)
for term, suggestions in nested.items():
print(f"{term}: {suggestions}")
```
---
## HTTP Clients
The library supports 7 HTTP clients with automatic fallback. If one fails, it tries the next.
### Supported Clients
1. **requests** (default) - Standard Python HTTP library
2. **curl_cffi** - cURL with browser impersonation
3. **tls_client** - Advanced TLS fingerprinting
4. **httpx** - Modern async-capable HTTP client
5. **urllib3** - Low-level HTTP client
6. **cloudscraper** - Cloudflare bypass
7. **aiohttp** - Async HTTP client
### Usage
```python
# Default (tries requests first, then others)
scraper = GPlayScraper()
# Specify a client
scraper = GPlayScraper(http_client="curl_cffi")
scraper = GPlayScraper(http_client="tls_client")
scraper = GPlayScraper(http_client="httpx")
```
### Installation
```bash
# Default
pip install requests
# Advanced clients (optional)
pip install curl-cffi
pip install tls-client
pip install httpx
pip install urllib3
pip install cloudscraper
pip install aiohttp
```
**Note:** The library automatically falls back to available clients if your preferred one fails.
---
## Methods
### `suggest_analyze(term, count=5, lang='en', country='us')`
Returns search suggestions as a list of strings.
```python
suggestions = scraper.suggest_analyze("video", count=5)
# Returns: ['video player', 'video editor', 'video downloader', 'video maker', 'video call']
```
### `suggest_nested(term, count=5, lang='en', country='us')`
Returns nested suggestions (suggestions for each suggestion).
```python
nested = scraper.suggest_nested("video", count=3)
# Returns: {
# 'video player': ['video player hd', 'video player all format', 'video player pro'],
# 'video editor': ['video editor pro', 'video editor free', 'video editor app'],
# 'video downloader': ['video downloader for facebook', 'video downloader hd', ...]
# }
```
### `suggest_print_all(term, count=5, lang='en', country='us')`
Prints suggestions as formatted JSON.
```python
scraper.suggest_print_all("video", count=5)
# Output: ["video player", "video editor", "video downloader", "video maker", "video call"]
```
### `suggest_print_nested(term, count=5, lang='en', country='us')`
Prints nested suggestions as formatted JSON.
```python
scraper.suggest_print_nested("video", count=3)
# Output: Full JSON object with nested suggestions
```
---
## Return Formats
### Simple Suggestions (List)
```python
['video player', 'video editor', 'video downloader', 'video maker', 'video call']
```
### Nested Suggestions (Dictionary)
```python
{
'video player': ['video player hd', 'video player all format', 'video player pro'],
'video editor': ['video editor pro', 'video editor free', 'video editor app'],
'video downloader': ['video downloader for facebook', 'video downloader hd']
}
```
---
## Practical Examples
### Autocomplete Feature
```python
def autocomplete(user_input):
"""Provide autocomplete suggestions as user types"""
if len(user_input) < 2:
return []
suggestions = scraper.suggest_analyze(user_input, count=10)
return suggestions
# Usage
print(autocomplete("gam")) # ['game', 'games', 'gaming', ...]
print(autocomplete("photo")) # ['photo editor', 'photo collage', ...]
```
### Keyword Research
```python
base_keywords = ["fitness", "workout", "exercise"]
all_keywords = set()
for keyword in base_keywords:
suggestions = scraper.suggest_analyze(keyword, count=10)
all_keywords.update(suggestions)
print(f"{keyword}: {len(suggestions)} suggestions")
print(f"\nTotal unique keywords: {len(all_keywords)}")
print("Sample keywords:", list(all_keywords)[:10])
```
### Deep Keyword Mining
```python
term = "photo editor"
nested = scraper.suggest_nested(term, count=5)
print(f"Keyword tree for '{term}':")
for parent, children in nested.items():
print(f"\n{parent}:")
for child in children:
print(f" - {child}")
```
### ASO Keyword Discovery
```python
import json
def discover_keywords(seed_term, depth=2):
"""Discover keywords with specified depth"""
keywords = {}
# Level 1
level1 = scraper.suggest_analyze(seed_term, count=10)
keywords[seed_term] = level1
if depth > 1:
# Level 2
for term in level1[:5]: # Limit to avoid too many requests
level2 = scraper.suggest_analyze(term, count=5)
keywords[term] = level2
return keywords
keywords = discover_keywords("game", depth=2)
print(json.dumps(keywords, indent=2))
```
### Trending Search Terms
```python
categories = ["game", "social", "productivity", "photo", "music"]
trending = {}
for category in categories:
suggestions = scraper.suggest_analyze(category, count=5)
trending[category] = suggestions
print(f"{category}: {', '.join(suggestions[:3])}...")
```
### Long-Tail Keywords
```python
short_term = "vpn"
suggestions = scraper.suggest_analyze(short_term, count=10)
# Filter for long-tail (3+ words)
long_tail = [s for s in suggestions if len(s.split()) >= 3]
print(f"Long-tail keywords for '{short_term}':")
for keyword in long_tail:
print(f"- {keyword}")
```
### Competitor Keyword Analysis
```python
competitor_apps = ["whatsapp", "telegram", "signal"]
all_suggestions = {}
for app in competitor_apps:
suggestions = scraper.suggest_analyze(app, count=10)
all_suggestions[app] = suggestions
print(f"{app}: {len(suggestions)} suggestions")
# Find common keywords
common = set(all_suggestions[competitor_apps[0]])
for app in competitor_apps[1:]:
common &= set(all_suggestions[app])
print(f"\nCommon keywords: {common}")
```
### Export Keyword Map
```python
import json
term = "fitness"
nested = scraper.suggest_nested(term, count=10)
with open(f'keywords_{term}.json', 'w') as f:
json.dump(nested, f, indent=2)
print(f"Exported keyword map for '{term}'")
print(f"Total parent keywords: {len(nested)}")
print(f"Total child keywords: {sum(len(v) for v in nested.values())}")
```
### Search Volume Estimation
```python
term = "photo editor"
suggestions = scraper.suggest_analyze(term, count=20)
# Suggestions appear in order of popularity (roughly)
print(f"Top suggestions for '{term}' (by estimated popularity):")
for i, suggestion in enumerate(suggestions[:10], 1):
print(f"{i}. {suggestion}")
```
---
## Parameters
### Initialization
- `http_client` (str, optional) - HTTP client to use: "requests", "curl_cffi", "tls_client", "httpx", "urllib3", "cloudscraper", "aiohttp" (default: "requests")
### Method Parameters
- `term` (str, required) - Search term or keyword
- `count` (int, optional) - Number of suggestions to return (default: 5, max: ~10)
- `lang` (str, optional) - Language code (default: 'en')
- `country` (str, optional) - Country code (default: 'us')
### Search Term Tips
- Use partial words: "gam" → "game", "games", "gaming"
- Try categories: "fitness", "photo", "music"
- Test variations: "vpn", "vpn free", "vpn app"
- Use brand names: "whatsapp", "instagram"
- Combine terms: "photo editor free"
### Language & Country Codes
- **Language**: 'en', 'es', 'fr', 'de', 'ja', 'ko', 'pt', 'ru', 'zh', etc.
- **Country**: 'us', 'gb', 'ca', 'au', 'in', 'br', 'jp', 'kr', 'de', 'fr', etc.
---
## When to Use Each Method
- **`suggest_analyze()`** - Get simple list of suggestions for autocomplete or keyword research
- **`suggest_nested()`** - Deep keyword mining with two levels of suggestions
- **`suggest_print_all()`** - Quick debugging/console output of suggestions
- **`suggest_print_nested()`** - Quick debugging/console output of nested suggestions
---
## Use Cases
### App Store Optimization (ASO)
- Discover high-traffic keywords
- Find long-tail keyword opportunities
- Analyze competitor keywords
- Optimize app title and description
### Market Research
- Identify trending search terms
- Understand user search behavior
- Discover niche markets
- Track keyword trends over time
### Content Strategy
- Generate content ideas
- Find related topics
- Optimize metadata
- Improve discoverability
### Competitive Analysis
- Discover competitor keywords
- Find keyword gaps
- Identify market opportunities
- Track competitor positioning
---
## Advanced Features
### Rate Limiting
Built-in rate limiting (1 second delay between requests) prevents blocking.
### Error Handling
```python
from gplay_scraper import GPlayScraper, NetworkError
scraper = GPlayScraper()
try:
suggestions = scraper.suggest_analyze("")
except ValueError:
print("Term cannot be empty")
except NetworkError:
print("Network error occurred")
```
### Multi-Region Suggestions
```python
# Get suggestions from different regions
us_suggestions = scraper.suggest_analyze("game", country="us")
uk_suggestions = scraper.suggest_analyze("game", country="gb")
jp_suggestions = scraper.suggest_analyze("game", country="jp", lang="ja")
print(f"US: {us_suggestions[:3]}")
print(f"UK: {uk_suggestions[:3]}")
print(f"JP: {jp_suggestions[:3]}")
```
### Batch Processing
```python
terms = ["fitness", "diet", "workout", "yoga", "meditation"]
all_suggestions = {}
for term in terms:
suggestions = scraper.suggest_analyze(term, count=10)
all_suggestions[term] = suggestions
print(f"{term}: {len(suggestions)} suggestions")
# Find overlapping keywords
all_keywords = set()
for suggestions in all_suggestions.values():
all_keywords.update(suggestions)
print(f"\nTotal unique keywords: {len(all_keywords)}")
```
### Recursive Keyword Expansion
```python
def expand_keywords(term, max_depth=2, current_depth=0):
"""Recursively expand keywords"""
if current_depth >= max_depth:
return []
suggestions = scraper.suggest_analyze(term, count=5)
all_keywords = suggestions.copy()
if current_depth < max_depth - 1:
for suggestion in suggestions[:2]: # Limit to avoid explosion
child_keywords = expand_keywords(suggestion, max_depth, current_depth + 1)
all_keywords.extend(child_keywords)
return all_keywords
keywords = expand_keywords("game", max_depth=2)
print(f"Expanded to {len(set(keywords))} unique keywords")
```
### Suggestion Filtering
```python
term = "game"
suggestions = scraper.suggest_analyze(term, count=20)
# Filter by length
short = [s for s in suggestions if len(s.split()) <= 2]
long = [s for s in suggestions if len(s.split()) > 2]
# Filter by keyword
free_games = [s for s in suggestions if 'free' in s.lower()]
print(f"Short keywords: {len(short)}")
print(f"Long keywords: {len(long)}")
print(f"Free games: {len(free_games)}")
```
================================================
FILE: README.md
================================================
# Google Play Scraper - Python Library 📱
[](https://badge.fury.io/py/gplay-scraper)
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
[](https://mohammedcha.github.io/gplay-scraper/)
[](https://pepy.tech/project/gplay-scraper)
[](https://github.com/Mohammedcha/gplay-scraper/stargazers)
[](https://github.com/Mohammedcha/gplay-scraper/issues)
**GPlay Scraper** is a powerful Python library for extracting comprehensive data from the Google Play Store. Built for developers, data analysts, and researchers, it provides easy access to app information, user reviews, search results, top charts, and market intelligence—all without requiring API keys.
## 🐛 Found a Bug? Help Us Improve!
**We value your feedback!** If you encounter any bugs, errors, or have suggestions for improvements, please [open an issue](https://github.com/Mohammedcha/gplay-scraper/issues). Contributors who report bugs or suggest features will be acknowledged in our [Contributors section](#-contributors) 🙏
## 🎯 What Can You Scrape?
**App Data (65+ Fields)**
- Basic info: title, developer, description, category, genre
- Ratings & reviews: score, ratings count, histogram, user reviews
- Install metrics: install count ranges, download statistics
- Pricing: free/paid status, price, in-app purchases, currency
- Media: icon, screenshots, video, header image URLs
- Technical: version, size, Android version, release date, last update
- Content: age rating, privacy policy, developer contact info
- Features: permissions, what's new, developer website
**Search & Discovery**
- Search apps by keyword with filtering and pagination
- Get search suggestions and autocomplete terms
- Find similar/competitor apps for any app
- Access top charts (free, paid, grossing) across 54 categories
**Developer Intelligence**
- Get complete app portfolio for any developer
- Track developer's app performance and ratings
- Analyze developer's market presence
**User Reviews**
- Extract reviews with ratings, text, and timestamps
- Get reviewer names and helpful vote counts
- Filter by newest, most relevant, or highest rated
- Track app versions mentioned in reviews
**Market Research**
- Multi-language support (100+ languages)
- Multi-region data (150+ countries)
- Localized pricing and availability
- Competitive analysis and benchmarking
## 🆕 **What's New in v1.0.6**
**✅ Critical Bug Fixes:**
- **Reviews Pagination Fix** - Fixed critical issue when requesting more reviews than available
- **NoneType Error Resolution** - Resolved 'NoneType' object is not subscriptable error in reviews
- **Empty Response Handling** - Better handling of apps with limited reviews
- **Token Extraction Logic** - Improved pagination token handling for empty responses
- **Graceful Degradation** - Now returns available reviews instead of crashing
**✅ Enhanced Reliability:**
- **Safe Bounds Checking** - Added proper bounds checking for pagination tokens
- **Null Checking** - Enhanced null checking for empty data structures
- **Error Recovery** - Improved error handling in ReviewsScraper and ReviewsParser
- **Stability Improvements** - Better handling of edge cases in reviews extraction
**🙏 Acknowledgments:**
- Thanks to [@PhamDinhThienVu](https://github.com/PhamDinhThienVu) for reporting the reviews pagination bug
**✅ 7 Method Types:**
- **App Methods** - Extract 65+ data fields from any app (ratings, installs, pricing, permissions, etc.)
- **Search Methods** - Search Google Play Store apps with comprehensive filtering
- **Reviews Methods** - Extract user reviews with ratings, timestamps, and detailed feedback
- **Developer Methods** - Get all apps published by a specific developer
- **List Methods** - Access top charts (top free, top paid, top grossing) by category
- **Similar Methods** - Find similar/competitor apps for market research
- **Suggest Methods** - Get search suggestions and autocomplete for ASO
## ⚡ Key Features
**Powerful & Flexible**
- **7 HTTP clients with automatic fallback** - requests, curl_cffi, tls_client, httpx, urllib3, cloudscraper, aiohttp
- **42 functions across 7 method types** - analyze(), get_field(), get_fields(), print_field(), print_fields(), print_all()
- **No API keys required** - Direct scraping from Google Play Store
- **Multi-language & multi-region** - 100+ languages, 150+ countries
**Reliable & Safe**
- **Built-in rate limiting** - Prevents blocking with automatic delays
- **Automatic HTTP client fallback** - Ensures maximum reliability
- **Error handling** - Graceful failures with informative messages
- **Retry logic** - Automatic retries for failed requests
**Developer Friendly**
- **Simple API** - Intuitive method names and parameters
- **Comprehensive documentation** - Examples for every use case
- **Type hints** - Full IDE autocomplete support
- **Flexible output** - Get data as dict/list or print as JSON
## 📋 Requirements
- Python 3.7+
- requests (default HTTP client)
- Optional: curl-cffi, tls-client, httpx, urllib3, cloudscraper, aiohttp (for advanced HTTP clients)
## 🚀 Installation
```bash
# Install from PyPI
pip install gplay-scraper
# Or install in development mode
pip install -e .
```
## 📖 Quick Start
```python
from gplay_scraper import GPlayScraper
# Initialize with HTTP client (curl_cffi recommended for best performance)
scraper = GPlayScraper(http_client="curl_cffi")
# Get app details with different image sizes
app_id = "com.whatsapp"
scraper.app_print_all(app_id, lang="en", country="us", assets="LARGE")
# Get high-quality app data
data = scraper.app_analyze(app_id, assets="ORIGINAL") # Maximum image quality
icon_small = scraper.app_get_field(app_id, "icon", assets="SMALL") # 512px icon
# Print specific fields with custom image sizes
scraper.app_print_field(app_id, "icon", assets="LARGE") # Print large icon URL
scraper.app_print_fields(app_id, ["icon", "screenshots"], assets="ORIGINAL") # Print multiple fields
# Search for apps
scraper.search_print_all("social media", count=10, lang="en", country="us")
# Get reviews
scraper.reviews_print_all(app_id, count=50, sort="NEWEST", lang="en", country="us")
# Get developer apps
scraper.developer_print_all("5700313618786177705", count=20, lang="en", country="us")
# Get top charts
scraper.list_print_all("TOP_FREE", "GAME", count=20, lang="en", country="us")
# Get similar apps
scraper.similar_print_all(app_id, count=30, lang="en", country="us")
# Get search suggestions
scraper.suggest_print_all("fitness", count=5, lang="en", country="us")
```
## 🎯 7 Method Types
GPlay Scraper provides 7 method types with 42 functions to interact with Google Play Store data:
### 1. [App Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/APP_METHODS.md) - Extract app details (65+ fields)
### 2. [Search Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SEARCH_METHODS.md) - Search for apps by keyword
### 3. [Reviews Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/REVIEWS_METHODS.md) - Get user reviews and ratings
### 4. [Developer Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/DEVELOPER_METHODS.md) - Get all apps from a developer
### 5. [List Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/LIST_METHODS.md) - Get top charts (free, paid, grossing)
### 6. [Similar Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SIMILAR_METHODS.md) - Find similar/related apps
### 7. [Suggest Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SUGGEST_METHODS.md) - Get search suggestions/autocomplete
Each method type has 6 functions:
- `analyze()` - Get all data as dictionary/list
- `get_field()` - Get single field value
- `get_fields()` - Get multiple fields
- `print_field()` - Print single field to console
- `print_fields()` - Print multiple fields to console
- `print_all()` - Print all data as JSON
## 🎯 Method Examples
### 1. [App Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/APP_METHODS.md) - Get App Details
Extract comprehensive information about any app including ratings, installs, pricing, and 65+ data fields.
📖 **[View detailed documentation →](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/APP_METHODS.md)**
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper(http_client="curl_cffi")
# Print all app data as JSON
scraper.app_print_all("com.whatsapp", lang="en", country="us")
```
**What you get:** Complete app profile with title, developer, ratings, install counts, pricing, screenshots, permissions, and more.
📄 **[View JSON example →](https://github.com/Mohammedcha/gplay-scraper/blob/main/output/app_example.json)**
---
### 2. [Search Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SEARCH_METHODS.md) - Find Apps by Keyword
Search the Play Store by keyword, app name, or category to discover apps.
📖 **[View detailed documentation →](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SEARCH_METHODS.md)**
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper(http_client="curl_cffi")
# Print all search results as JSON
scraper.search_print_all("fitness tracker", count=20, lang="en", country="us")
```
**What you get:** List of apps matching your search with titles, developers, ratings, prices, and Play Store URLs.
📄 **[View JSON example →](https://github.com/Mohammedcha/gplay-scraper/blob/main/output/search_example.json)**
---
### 3. [Reviews Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/REVIEWS_METHODS.md) - Extract User Reviews
Get user reviews with ratings, comments, timestamps, and helpful votes for sentiment analysis.
📖 **[View detailed documentation →](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/REVIEWS_METHODS.md)**
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper(http_client="curl_cffi")
# Print all reviews as JSON
scraper.reviews_print_all("com.whatsapp", count=100, sort="NEWEST", lang="en", country="us")
```
**What you get:** User reviews with names, ratings (1-5 stars), review text, timestamps, app versions, and helpful vote counts.
📄 **[View JSON example →](https://github.com/Mohammedcha/gplay-scraper/blob/main/output/reviews_example.json)**
---
### 4. [Developer Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/DEVELOPER_METHODS.md) - Get Developer's Apps
Retrieve all apps published by a specific developer using their developer ID.
📖 **[View detailed documentation →](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/DEVELOPER_METHODS.md)**
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper(http_client="curl_cffi")
# Print all developer apps as JSON
scraper.developer_print_all("5700313618786177705", count=50, lang="en", country="us")
```
**What you get:** Complete portfolio of apps from a developer with titles, ratings, prices, and descriptions.
📄 **[View JSON example →](https://github.com/Mohammedcha/gplay-scraper/blob/main/output/developer_example.json)**
---
### 5. [List Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/LIST_METHODS.md) - Get Top Charts
Access Play Store top charts including top free, top paid, and top grossing apps by category.
📖 **[View detailed documentation →](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/LIST_METHODS.md)**
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper(http_client="curl_cffi")
# Print top free games as JSON
scraper.list_print_all("TOP_FREE", "GAME", count=50, lang="en", country="us")
```
**What you get:** Top-ranked apps with titles, developers, ratings, install counts, prices, and screenshots.
📄 **[View JSON example →](https://github.com/Mohammedcha/gplay-scraper/blob/main/output/list_example.json)**
---
### 6. [Similar Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SIMILAR_METHODS.md) - Find Related Apps
Discover apps similar to a reference app for competitive analysis and market research.
📖 **[View detailed documentation →](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SIMILAR_METHODS.md)**
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper(http_client="curl_cffi")
# Print similar apps as JSON
scraper.similar_print_all("com.whatsapp", count=30, lang="en", country="us")
```
**What you get:** List of similar/competitor apps with titles, developers, ratings, and pricing information.
📄 **[View JSON example →](https://github.com/Mohammedcha/gplay-scraper/blob/main/output/similar_example.json)**
---
### 7. [Suggest Methods](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SUGGEST_METHODS.md) - Get Search Suggestions
Get autocomplete suggestions and keyword ideas for ASO and market research.
📖 **[View detailed documentation →](https://github.com/Mohammedcha/gplay-scraper/blob/main/README/SUGGEST_METHODS.md)**
```python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper(http_client="curl_cffi")
# Print search suggestions as JSON
scraper.suggest_print_all("photo editor", count=10, lang="en", country="us")
```
**What you get:** List of popular search terms related to your keyword for ASO and keyword research.
📄 **[View JSON example →](https://github.com/Mohammedcha/gplay-scraper/blob/main/output/suggest_example.json)**
---
## 🤝 Contributing
1. Fork the repository
2. Create your feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request
## 📄 License
This project is licensed under the MIT License.
## 🙏 Contributors
Special thanks to developers who helped improve this library:
- [@PhamDinhThienVu](https://github.com/PhamDinhThienVu) - Reported reviews pagination bug (v1.0.6)
- [@elmissouri16](https://github.com/elmissouri16) - Suggested multiple HTTP clients support (v1.0.3)
---
**Happy Analyzing! 🚀**
================================================
FILE: SECURITY.md
================================================
# Security Policy
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 1.0.x | :white_check_mark: |
## Reporting a Vulnerability
If you discover a security vulnerability, please report it through GitHub Issues with the "security" label.
Please do not report security vulnerabilities through public GitHub issues.
We will respond to security reports within 48 hours.
================================================
FILE: build_docs.py
================================================
#!/usr/bin/env python3
"""Build documentation using Sphinx."""
import os
import sys
import subprocess
from pathlib import Path
def install_docs_requirements():
"""Install documentation requirements."""
print("[INFO] Installing documentation requirements...")
try:
subprocess.run([
sys.executable, "-m", "pip", "install",
"-r", "docs/requirements.txt"
], check=True)
print("[OK] Documentation requirements installed!")
except subprocess.CalledProcessError as e:
print(f"[ERROR] Failed to install requirements: {e}")
return False
return True
def build_html_docs():
"""Build HTML documentation."""
print("[INFO] Building HTML documentation...")
docs_dir = Path("docs")
build_dir = docs_dir / "_build" / "html"
try:
# Change to docs directory
os.chdir(docs_dir)
# Build documentation
subprocess.run([
"sphinx-build", "-b", "html", ".", "_build/html"
], check=True)
print("[OK] Documentation built successfully!")
print(f"[INFO] Open: {Path('_build/html/index.html').absolute()}")
return True
except subprocess.CalledProcessError as e:
print(f"[ERROR] Failed to build documentation: {e}")
return False
except FileNotFoundError:
print("[ERROR] Sphinx not found. Installing requirements first...")
return False
def main():
"""Main function to build documentation."""
print("=== GPlay Scraper Documentation Builder ===\n")
# Install requirements
if not install_docs_requirements():
return 1
# Build documentation
if not build_html_docs():
return 1
print("\n[SUCCESS] Documentation build complete!")
print("[INFO] Open docs/_build/html/index.html in your browser")
return 0
if __name__ == "__main__":
sys.exit(main())
================================================
FILE: docs/README.md
================================================
# GPlay Scraper Documentation
## Build Documentation
```bash
pip install -r requirements.txt
cd docs
sphinx-build -b html . _build/html
```
## Open Documentation
```bash
start _build/html/index.html # Windows
open _build/html/index.html # Mac
xdg-open _build/html/index.html # Linux
```
## Live Reload
```bash
pip install sphinx-autobuild
sphinx-autobuild . _build/html
```
================================================
FILE: docs/api/app.rst
================================================
App Methods
===========
Extract comprehensive app data with 57 fields including install analytics, ratings, pricing, and developer information.
Overview
--------
The App methods provide access to detailed information about any Google Play Store app. All methods return data in JSON format with 57 fields.
Available Methods
-----------------
* ``app_analyze()`` - Get all 57 fields
* ``app_get_field()`` - Get single field value
* ``app_get_fields()`` - Get multiple field values
* ``app_print_field()`` - Print single field
* ``app_print_fields()`` - Print multiple fields
* ``app_print_all()`` - Print all data as JSON
app_analyze()
-------------
Get complete app data with all 57 fields.
**Signature:**
.. code-block:: python
app_analyze(app_id, lang='en', country='', assets=None)
**Parameters:**
* ``app_id`` (str, required) - Google Play app ID (e.g., 'com.whatsapp')
* ``lang`` (str, optional) - Language code (default: 'en')
* ``country`` (str, optional) - Country code (default: '')
* ``assets`` (str, optional) - Image size: 'SMALL', 'MEDIUM', 'LARGE', 'ORIGINAL'
**Returns:**
Dictionary with 57 fields
**Example:**
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
app = scraper.app_analyze('com.whatsapp')
print(app['title']) # WhatsApp Messenger
print(app['developer']) # WhatsApp LLC
print(app['score']) # 4.2189474
print(app['realInstalls']) # 10931553905
print(app['dailyInstalls']) # 1815870
print(app['publisherCountry']) # United States
**Multi-language Example:**
.. code-block:: python
# Get app data in Spanish
app_es = scraper.app_analyze('com.whatsapp', lang='es')
print(app_es['description']) # Description in Spanish
# Get app data for UK region
app_uk = scraper.app_analyze('com.whatsapp', country='gb')
**Image Size Example:**
.. code-block:: python
# Get large images
app = scraper.app_analyze('com.whatsapp', assets='LARGE')
print(app['icon']) # URL with =w2048 parameter
app_get_field()
---------------
Get a single field value from app data.
**Signature:**
.. code-block:: python
app_get_field(app_id, field, lang='en', country='', assets=None)
**Parameters:**
* ``app_id`` (str, required) - Google Play app ID
* ``field`` (str, required) - Field name to retrieve
* ``lang`` (str, optional) - Language code
* ``country`` (str, optional) - Country code
* ``assets`` (str, optional) - Image size
**Returns:**
Value of the requested field (type depends on field)
**Example:**
.. code-block:: python
scraper = GPlayScraper()
# Get title
title = scraper.app_get_field('com.whatsapp', 'title')
print(title) # "WhatsApp Messenger"
# Get score
score = scraper.app_get_field('com.whatsapp', 'score')
print(score) # 4.2189474
# Get daily installs
daily = scraper.app_get_field('com.whatsapp', 'dailyInstalls')
print(f"{daily:,}") # 1,815,870
app_get_fields()
----------------
Get multiple field values from app data.
**Signature:**
.. code-block:: python
app_get_fields(app_id, fields, lang='en', country='', assets=None)
**Parameters:**
* ``app_id`` (str, required) - Google Play app ID
* ``fields`` (list, required) - List of field names
* ``lang`` (str, optional) - Language code
* ``country`` (str, optional) - Country code
* ``assets`` (str, optional) - Image size
**Returns:**
Dictionary with requested fields and values
**Example:**
.. code-block:: python
scraper = GPlayScraper()
fields = ['title', 'developer', 'score', 'realInstalls', 'dailyInstalls']
data = scraper.app_get_fields('com.whatsapp', fields)
print(data)
# {
# 'title': 'WhatsApp Messenger',
# 'developer': 'WhatsApp LLC',
# 'score': 4.2189474,
# 'realInstalls': 10931553905,
# 'dailyInstalls': 1815870
# }
app_print_field()
-----------------
Print a single field value to console.
**Signature:**
.. code-block:: python
app_print_field(app_id, field, lang='en', country='', assets=None)
**Returns:**
None (prints to console)
**Example:**
.. code-block:: python
scraper = GPlayScraper()
scraper.app_print_field('com.whatsapp', 'title')
# Output: title: WhatsApp Messenger
app_print_fields()
------------------
Print multiple field values to console.
**Signature:**
.. code-block:: python
app_print_fields(app_id, fields, lang='en', country='', assets=None)
**Returns:**
None (prints to console)
**Example:**
.. code-block:: python
scraper = GPlayScraper()
fields = ['title', 'score', 'dailyInstalls']
scraper.app_print_fields('com.whatsapp', fields)
app_print_all()
---------------
Print all 57 fields as formatted JSON to console.
**Signature:**
.. code-block:: python
app_print_all(app_id, lang='en', country='', assets=None)
**Returns:**
None (prints to console)
**Example:**
.. code-block:: python
scraper = GPlayScraper()
scraper.app_print_all('com.whatsapp')
# Outputs all 57 fields as formatted JSON
Available Fields
----------------
The App methods return 57 fields organized into categories:
**Basic Information (5 fields)**
* ``appId`` - Package identifier
* ``title`` - App name
* ``summary`` - Short description
* ``description`` - Full description
* ``appUrl`` - Play Store URL
**Category (4 fields)**
* ``genre`` - Primary category
* ``genreId`` - Category ID
* ``categories`` - All categories (array)
* ``available`` - Availability status (boolean)
**Release & Updates (3 fields)**
* ``released`` - Release date
* ``appAgeDays`` - Days since release (computed)
* ``lastUpdated`` - Last update date
**Media (5 fields)**
* ``icon`` - App icon URL
* ``headerImage`` - Header image URL
* ``screenshots`` - Screenshot URLs (array)
* ``video`` - Promotional video URL
* ``videoImage`` - Video thumbnail URL
**Install Statistics (10 fields)**
* ``installs`` - Install range string
* ``minInstalls`` - Minimum installs
* ``realInstalls`` - Exact install count
* ``dailyInstalls`` - Average daily installs (computed)
* ``minDailyInstalls`` - Min daily installs (computed)
* ``realDailyInstalls`` - Real daily installs (computed)
* ``monthlyInstalls`` - Average monthly installs (computed)
* ``minMonthlyInstalls`` - Min monthly installs (computed)
* ``realMonthlyInstalls`` - Real monthly installs (computed)
**Ratings (4 fields)**
* ``score`` - Average rating (0-5)
* ``ratings`` - Total number of ratings
* ``reviews`` - Total number of reviews
* ``histogram`` - Rating distribution [1★, 2★, 3★, 4★, 5★]
**Ads (2 fields)**
* ``adSupported`` - Supports ads (boolean)
* ``containsAds`` - Contains ads (boolean)
**Technical (7 fields)**
* ``version`` - Current version
* ``androidVersion`` - Minimum Android version
* ``maxAndroidApi`` - Maximum Android API level
* ``minAndroidApi`` - Minimum Android API level
* ``appBundle`` - Bundle identifier
* ``contentRating`` - Age rating
* ``contentRatingDescription`` - Rating description
**Updates (1 field)**
* ``whatsNew`` - Changelog (array)
**Privacy (2 fields)**
* ``permissions`` - Required permissions (object)
* ``dataSafety`` - Data safety info (array)
**Pricing (7 fields)**
* ``price`` - App price
* ``currency`` - Currency code
* ``free`` - Is free (boolean)
* ``offersIAP`` - Has in-app purchases (boolean)
* ``inAppProductPrice`` - IAP price range
* ``sale`` - On sale (boolean)
* ``originalPrice`` - Original price if on sale
**Developer (8 fields)**
* ``developer`` - Developer name
* ``developerId`` - Developer ID
* ``developerEmail`` - Contact email
* ``developerWebsite`` - Website URL
* ``developerAddress`` - Physical address
* ``developerPhone`` - Contact phone
* ``publisherCountry`` - Publisher country (computed)
* ``privacyPolicy`` - Privacy policy URL
Common Use Cases
----------------
App Analytics
^^^^^^^^^^^^^
.. code-block:: python
scraper = GPlayScraper()
app = scraper.app_analyze('com.whatsapp')
print(f"App: {app['title']}")
print(f"Total Installs: {app['realInstalls']:,}")
print(f"Daily Installs: {app['dailyInstalls']:,}")
print(f"Monthly Installs: {app['monthlyInstalls']:,}")
print(f"Age: {app['appAgeDays']} days")
print(f"Rating: {app['score']}/5 ({app['ratings']:,} ratings)")
Competitor Comparison
^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
apps = ['com.whatsapp', 'org.telegram.messenger', 'org.thoughtcrime.securesms']
for app_id in apps:
app = scraper.app_analyze(app_id)
print(f"{app['title']}")
print(f" Installs: {app['realInstalls']:,}")
print(f" Rating: {app['score']}/5")
print(f" Daily Growth: {app['dailyInstalls']:,}")
Market Research
^^^^^^^^^^^^^^^
.. code-block:: python
# Get key metrics for analysis
fields = [
'title', 'developer', 'score', 'ratings',
'realInstalls', 'dailyInstalls', 'free', 'price'
]
apps = ['com.app1', 'com.app2', 'com.app3']
for app_id in apps:
data = scraper.app_get_fields(app_id, fields)
print(data)
See Also
--------
* :doc:`../fields` - Complete field reference
* :doc:`../examples` - More practical examples
* :doc:`../configuration` - Configuration options
================================================
FILE: docs/api/developer.rst
================================================
Developer Methods
=================
Get all apps from a specific developer or company.
Overview
--------
The Developer methods allow you to find all apps published by a developer, returning 11 fields per app.
Available Methods
-----------------
* ``developer_analyze()`` - Get all developer apps
* ``developer_get_field()`` - Get single field from all apps
* ``developer_get_fields()`` - Get multiple fields from all apps
* ``developer_print_field()`` - Print single field
* ``developer_print_fields()`` - Print multiple fields
* ``developer_print_all()`` - Print all apps as JSON
developer_analyze()
-------------------
Get all apps from a developer.
**Signature:**
.. code-block:: python
developer_analyze(dev_id, count=100, lang='en', country='')
**Parameters:**
* ``dev_id`` (str, required) - Developer name or numeric ID
* ``count`` (int, optional) - Number of apps (default: 100)
* ``lang`` (str, optional) - Language code
* ``country`` (str, optional) - Country code
**Returns:**
List of dictionaries, each with 11 fields
**Example:**
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Using developer name
apps = scraper.developer_analyze('Google LLC')
for app in apps:
print(f"{app['title']}")
print(f" Developer: {app['developer']}")
print(f" Rating: {app['score']}/5")
print(f" Free: {app['free']}")
**Using Numeric Developer ID:**
.. code-block:: python
# Using numeric ID
apps = scraper.developer_analyze('5700313618786177705')
Available Fields
----------------
Each app contains 11 fields:
* ``appId`` - Package identifier
* ``title`` - App name
* ``url`` - Play Store URL
* ``icon`` - App icon URL
* ``developer`` - Developer name
* ``description`` - App description
* ``score`` - Average rating (0-5)
* ``scoreText`` - Rating as text
* ``price`` - App price
* ``free`` - Is free (boolean)
* ``currency`` - Currency code
Common Use Cases
----------------
Portfolio Analysis
^^^^^^^^^^^^^^^^^^
.. code-block:: python
scraper = GPlayScraper()
apps = scraper.developer_analyze('Google LLC')
# Calculate average rating
avg_rating = sum(app['score'] for app in apps) / len(apps)
# Count free vs paid
free_count = sum(1 for app in apps if app['free'])
paid_count = len(apps) - free_count
print(f"Total apps: {len(apps)}")
print(f"Average rating: {avg_rating:.2f}/5")
print(f"Free apps: {free_count}, Paid apps: {paid_count}")
Competitive Analysis
^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
developers = ['Google LLC', 'Microsoft Corporation', 'Meta Platforms, Inc.']
for dev in developers:
apps = scraper.developer_analyze(dev)
high_rated = [app for app in apps if app['score'] >= 4.5]
print(f"\n{dev}:")
print(f" Total apps: {len(apps)}")
print(f" High-rated apps (4.5+): {len(high_rated)}")
See Also
--------
* :doc:`app` - Get detailed app information
* :doc:`similar` - Find similar apps
================================================
FILE: docs/api/list.rst
================================================
List Methods
============
Get top charts (top free, top paid, top grossing apps).
Overview
--------
The List methods access Play Store top charts, returning 14 fields per app.
list_analyze()
--------------
**Signature:**
.. code-block:: python
list_analyze(collection='TOP_FREE', category='APPLICATION',
count=100, lang='en', country='')
**Parameters:**
* ``collection`` - 'TOP_FREE', 'TOP_PAID', 'TOP_GROSSING'
* ``category`` - App category (default: 'APPLICATION')
* ``count`` - Number of apps (default: 100, max: ~500)
**Example:**
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Top free games
top_free = scraper.list_analyze('TOP_FREE', category='GAME', count=100)
for i, app in enumerate(top_free, 1):
print(f"{i}. {app['title']} - {app['score']}/5")
Available Fields
----------------
14 fields including: title, appId, url, icon, screenshots, developer, genre, installs, description, score, scoreText, price, free, currency
Collections
-----------
* **TOP_FREE** - Top free apps
* **TOP_PAID** - Top paid apps
* **TOP_GROSSING** - Highest earning apps
App Categories (36)
-------------------
* ``APPLICATION`` - All apps (default)
* ``ANDROID_WEAR`` - Android Wear apps
* ``ART_AND_DESIGN`` - Art & design
* ``AUTO_AND_VEHICLES`` - Auto & vehicles
* ``BEAUTY`` - Beauty
* ``BOOKS_AND_REFERENCE`` - Books & reference
* ``BUSINESS`` - Business
* ``COMICS`` - Comics
* ``COMMUNICATION`` - Communication
* ``DATING`` - Dating
* ``EDUCATION`` - Education
* ``ENTERTAINMENT`` - Entertainment
* ``EVENTS`` - Events
* ``FINANCE`` - Finance
* ``FOOD_AND_DRINK`` - Food & drink
* ``HEALTH_AND_FITNESS`` - Health & fitness
* ``HOUSE_AND_HOME`` - House & home
* ``LIBRARIES_AND_DEMO`` - Libraries & demo
* ``LIFESTYLE`` - Lifestyle
* ``MAPS_AND_NAVIGATION`` - Maps & navigation
* ``MEDICAL`` - Medical
* ``MUSIC_AND_AUDIO`` - Music & audio
* ``NEWS_AND_MAGAZINES`` - News & magazines
* ``PARENTING`` - Parenting
* ``PERSONALIZATION`` - Personalization
* ``PHOTOGRAPHY`` - Photography
* ``PRODUCTIVITY`` - Productivity
* ``SHOPPING`` - Shopping
* ``SOCIAL`` - Social
* ``SPORTS`` - Sports
* ``TOOLS`` - Tools
* ``TRAVEL_AND_LOCAL`` - Travel & local
* ``VIDEO_PLAYERS`` - Video players & editors
* ``WATCH_FACE`` - Watch faces
* ``WEATHER`` - Weather
* ``FAMILY`` - Family
Game Categories (18)
---------------------
* ``GAME`` - All games
* ``GAME_ACTION`` - Action games
* ``GAME_ADVENTURE`` - Adventure games
* ``GAME_ARCADE`` - Arcade games
* ``GAME_BOARD`` - Board games
* ``GAME_CARD`` - Card games
* ``GAME_CASINO`` - Casino games
* ``GAME_CASUAL`` - Casual games
* ``GAME_EDUCATIONAL`` - Educational games
* ``GAME_MUSIC`` - Music games
* ``GAME_PUZZLE`` - Puzzle games
* ``GAME_RACING`` - Racing games
* ``GAME_ROLE_PLAYING`` - Role playing games
* ``GAME_SIMULATION`` - Simulation games
* ``GAME_SPORTS`` - Sports games
* ``GAME_STRATEGY`` - Strategy games
* ``GAME_TRIVIA`` - Trivia games
* ``GAME_WORD`` - Word games
Example
-------
.. code-block:: python
# Top paid communication apps
top_paid = scraper.list_analyze('TOP_PAID',
category='COMMUNICATION',
count=50)
================================================
FILE: docs/api/reviews.rst
================================================
Reviews Methods
===============
Extract user reviews and ratings with sorting options.
Overview
--------
The Reviews methods allow you to get user reviews with 8 fields per review, including content, rating, and metadata.
Available Methods
-----------------
* ``reviews_analyze()`` - Get all reviews
* ``reviews_get_field()`` - Get single field from all reviews
* ``reviews_get_fields()`` - Get multiple fields from all reviews
* ``reviews_print_field()`` - Print single field
* ``reviews_print_fields()`` - Print multiple fields
* ``reviews_print_all()`` - Print all reviews as JSON
reviews_analyze()
-----------------
Get user reviews with all details.
**Signature:**
.. code-block:: python
reviews_analyze(app_id, count=100, sort='NEWEST', lang='en', country='')
**Parameters:**
* ``app_id`` (str, required) - Google Play app ID
* ``count`` (int, optional) - Number of reviews (default: 100, max: ~1000+)
* ``sort`` (str, optional) - Sort order: 'NEWEST', 'RELEVANT', 'RATING' (default: 'NEWEST')
* ``lang`` (str, optional) - Language code
* ``country`` (str, optional) - Country code
**Returns:**
List of dictionaries, each with 8 fields
**Example:**
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Get newest reviews
reviews = scraper.reviews_analyze('com.whatsapp', count=50, sort='NEWEST')
for review in reviews:
print(f"{review['userName']}: {review['score']}/5")
print(f"Date: {review['at']}")
print(f"Content: {review['content'][:100]}...")
print(f"Helpful: {review['thumbsUpCount']} people")
Sort Options
^^^^^^^^^^^^
* **NEWEST** - Most recent reviews first (default)
* **RELEVANT** - Most helpful/relevant reviews
* **RATING** - Sorted by rating score
.. code-block:: python
# Get most relevant reviews
reviews = scraper.reviews_analyze('com.whatsapp',
count=100,
sort='RELEVANT')
# Get reviews sorted by rating
reviews = scraper.reviews_analyze('com.whatsapp',
count=100,
sort='RATING')
reviews_get_field()
-------------------
Get single field from all reviews.
**Example:**
.. code-block:: python
scraper = GPlayScraper()
# Get all usernames
usernames = scraper.reviews_get_field('com.whatsapp', 'userName', count=10)
# Get all scores
scores = scraper.reviews_get_field('com.whatsapp', 'score', count=100)
reviews_get_fields()
--------------------
Get multiple fields from all reviews.
**Example:**
.. code-block:: python
fields = ['userName', 'score', 'content', 'thumbsUpCount']
reviews = scraper.reviews_get_fields('com.whatsapp', fields, count=50)
Available Fields
----------------
Each review contains 8 fields:
* ``reviewId`` - Unique review identifier
* ``userName`` - Reviewer's name
* ``userImage`` - Reviewer's profile image URL
* ``content`` - Review text
* ``score`` - Rating (1-5)
* ``thumbsUpCount`` - Number of helpful votes
* ``at`` - Review date (ISO format)
* ``appVersion`` - App version reviewed
Common Use Cases
----------------
Monitor Negative Reviews
^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
scraper = GPlayScraper()
reviews = scraper.reviews_analyze('com.myapp', count=100, sort='NEWEST')
negative = [r for r in reviews if r['score'] <= 2]
print(f"Found {len(negative)} negative reviews")
for review in negative:
print(f"{review['userName']}: {review['score']}/5")
print(f" {review['content']}")
Sentiment Analysis
^^^^^^^^^^^^^^^^^^
.. code-block:: python
reviews = scraper.reviews_analyze('com.whatsapp', count=500)
# Count by rating
rating_counts = {1: 0, 2: 0, 3: 0, 4: 0, 5: 0}
for review in reviews:
rating_counts[review['score']] += 1
print("Rating distribution:")
for rating, count in rating_counts.items():
print(f" {rating}★: {count} reviews")
Find Helpful Reviews
^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
reviews = scraper.reviews_analyze('com.whatsapp',
count=100,
sort='RELEVANT')
# Get most helpful
most_helpful = sorted(reviews,
key=lambda x: x['thumbsUpCount'],
reverse=True)[:10]
for review in most_helpful:
print(f"{review['thumbsUpCount']} helpful votes")
print(f" {review['content'][:100]}...")
Track Version Feedback
^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
reviews = scraper.reviews_analyze('com.myapp', count=200)
# Group by app version
by_version = {}
for review in reviews:
version = review['appVersion']
if version not in by_version:
by_version[version] = []
by_version[version].append(review)
# Analyze each version
for version, version_reviews in by_version.items():
avg_score = sum(r['score'] for r in version_reviews) / len(version_reviews)
print(f"Version {version}: {avg_score:.2f}/5 ({len(version_reviews)} reviews)")
See Also
--------
* :doc:`app` - Get app information
* :doc:`../examples` - More examples
================================================
FILE: docs/api/search.rst
================================================
Search Methods
==============
Search for apps on Google Play Store by keyword and get results with 11 fields per app.
Overview
--------
The Search methods allow you to find apps by keyword, similar to using the Play Store search bar. Results include basic app information with 11 fields.
Available Methods
-----------------
* ``search_analyze()`` - Search and get all results
* ``search_get_field()`` - Get single field from all results
* ``search_get_fields()`` - Get multiple fields from all results
* ``search_print_field()`` - Print single field
* ``search_print_fields()`` - Print multiple fields
* ``search_print_all()`` - Print all results as JSON
search_analyze()
----------------
Search for apps and get complete results.
**Signature:**
.. code-block:: python
search_analyze(query, count=100, lang='en', country='')
**Parameters:**
* ``query`` (str, required) - Search query string
* ``count`` (int, optional) - Number of results to return (default: 100, max: ~250)
* ``lang`` (str, optional) - Language code (default: 'en')
* ``country`` (str, optional) - Country code (default: '')
**Returns:**
List of dictionaries, each with 11 fields
**Example:**
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
results = scraper.search_analyze('messaging', count=10)
for app in results:
print(f"{app['title']} by {app['developer']}")
print(f" Rating: {app['score']}/5")
print(f" Free: {app['free']}")
print(f" URL: {app['url']}")
**Pagination Example:**
.. code-block:: python
# Get top 100 results
results = scraper.search_analyze('games', count=100)
print(f"Found {len(results)} games")
search_get_field()
------------------
Get a single field from all search results.
**Signature:**
.. code-block:: python
search_get_field(query, field, count=100, lang='en', country='')
**Parameters:**
* ``query`` (str, required) - Search query
* ``field`` (str, required) - Field name to retrieve
* ``count`` (int, optional) - Number of results
* ``lang`` (str, optional) - Language code
* ``country`` (str, optional) - Country code
**Returns:**
List of field values from all results
**Example:**
.. code-block:: python
scraper = GPlayScraper()
# Get all titles
titles = scraper.search_get_field('messaging', 'title', count=10)
print(titles)
# ['WhatsApp Messenger', 'Telegram', 'Signal', ...]
# Get all ratings
scores = scraper.search_get_field('messaging', 'score', count=10)
print(scores)
# [4.2, 4.3, 4.5, ...]
search_get_fields()
-------------------
Get multiple fields from all search results.
**Signature:**
.. code-block:: python
search_get_fields(query, fields, count=100, lang='en', country='')
**Parameters:**
* ``query`` (str, required) - Search query
* ``fields`` (list, required) - List of field names
* ``count`` (int, optional) - Number of results
* ``lang`` (str, optional) - Language code
* ``country`` (str, optional) - Country code
**Returns:**
List of dictionaries with requested fields
**Example:**
.. code-block:: python
scraper = GPlayScraper()
fields = ['title', 'developer', 'score', 'free']
results = scraper.search_get_fields('games', fields, count=5)
for app in results:
print(f"{app['title']} - {app['score']}/5 - Free: {app['free']}")
search_print_field()
--------------------
Print single field from all search results.
**Signature:**
.. code-block:: python
search_print_field(query, field, count=100, lang='en', country='')
**Returns:**
None (prints to console)
search_print_fields()
---------------------
Print multiple fields from all search results.
**Signature:**
.. code-block:: python
search_print_fields(query, fields, count=100, lang='en', country='')
**Returns:**
None (prints to console)
search_print_all()
------------------
Print all search results as JSON.
**Signature:**
.. code-block:: python
search_print_all(query, count=100, lang='en', country='')
**Returns:**
None (prints to console)
Available Fields
----------------
Each search result contains 11 fields:
* ``title`` - App name
* ``appId`` - Package identifier
* ``url`` - Play Store URL
* ``icon`` - App icon URL
* ``developer`` - Developer name
* ``summary`` - Short description
* ``score`` - Average rating (0-5)
* ``scoreText`` - Rating as text (e.g., "4.2★")
* ``price`` - App price
* ``free`` - Is free (boolean)
* ``currency`` - Currency code
Common Use Cases
----------------
Find Apps by Category
^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
scraper = GPlayScraper()
# Find fitness apps
fitness_apps = scraper.search_analyze('fitness tracker', count=20)
# Filter by rating
high_rated = [app for app in fitness_apps if app['score'] >= 4.5]
for app in high_rated:
print(f"{app['title']}: {app['score']}/5")
Market Research
^^^^^^^^^^^^^^^
.. code-block:: python
# Research competitors
results = scraper.search_analyze('photo editor', count=50)
# Analyze free vs paid
free_apps = [app for app in results if app['free']]
paid_apps = [app for app in results if not app['free']]
print(f"Free apps: {len(free_apps)}")
print(f"Paid apps: {len(paid_apps)}")
# Average ratings
avg_free = sum(app['score'] for app in free_apps) / len(free_apps)
avg_paid = sum(app['score'] for app in paid_apps) / len(paid_apps)
print(f"Average free app rating: {avg_free:.2f}")
print(f"Average paid app rating: {avg_paid:.2f}")
Discovery
^^^^^^^^^
.. code-block:: python
# Discover trending apps
keywords = ['ai', 'chatbot', 'productivity']
for keyword in keywords:
results = scraper.search_analyze(keyword, count=5)
print(f"\nTop {keyword} apps:")
for app in results:
print(f" {app['title']} - {app['score']}/5")
Multi-Language Search
^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
# Search in Spanish
results_es = scraper.search_analyze('juegos', lang='es', count=10)
# Search in French
results_fr = scraper.search_analyze('jeux', lang='fr', count=10)
# Regional search (UK)
results_uk = scraper.search_analyze('games', country='gb', count=10)
See Also
--------
* :doc:`app` - Get detailed app information
* :doc:`../examples` - More practical examples
* :doc:`../configuration` - Configuration options
================================================
FILE: docs/api/similar.rst
================================================
Similar Methods
===============
Find apps similar to a given app (competitors or alternatives).
Overview
--------
The Similar methods help you discover competitor apps or alternatives, returning 11 fields per app.
similar_analyze()
-----------------
**Signature:**
.. code-block:: python
similar_analyze(app_id, count=100, lang='en', country='')
**Example:**
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
similar = scraper.similar_analyze('com.whatsapp', count=20)
for app in similar:
print(f"{app['title']} - {app['score']}/5")
Available Fields
----------------
Same 11 fields as Developer Methods.
Common Use Cases
----------------
Competitor Analysis
^^^^^^^^^^^^^^^^^^^
.. code-block:: python
# Find competitors
my_app = scraper.app_analyze('com.myapp')
competitors = scraper.similar_analyze('com.myapp', count=10)
print(f"My App: {my_app['score']}/5")
print("\nCompetitors:")
for comp in competitors:
print(f" {comp['title']}: {comp['score']}/5")
================================================
FILE: docs/api/suggest.rst
================================================
Suggest Methods
===============
Get search suggestions and autocomplete.
Overview
--------
The Suggest methods provide search suggestions, returning lists of strings.
suggest_analyze()
-----------------
Get search suggestions for a term.
**Signature:**
.. code-block:: python
suggest_analyze(term, count=5, lang='en', country='')
**Example:**
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
suggestions = scraper.suggest_analyze('mine', count=10)
print(suggestions)
# ['minecraft', 'minesweeper', 'mineplex', ...]
suggest_nested()
----------------
Get nested suggestions (suggestions for suggestions).
**Signature:**
.. code-block:: python
suggest_nested(term, count=5, lang='en', country='')
**Returns:**
Dictionary mapping terms to their suggestion lists
**Example:**
.. code-block:: python
nested = scraper.suggest_nested('game', count=5)
for term, suggestions in nested.items():
print(f"\n{term}:")
for suggestion in suggestions:
print(f" - {suggestion}")
Common Use Cases
----------------
Keyword Research
^^^^^^^^^^^^^^^^
.. code-block:: python
# Find popular search terms
keywords = ['fitness', 'photo', 'music']
for keyword in keywords:
suggestions = scraper.suggest_analyze(keyword, count=10)
print(f"\n{keyword} suggestions:")
for s in suggestions:
print(f" {s}")
Trend Discovery
^^^^^^^^^^^^^^^
.. code-block:: python
# Discover trending topics
term = "ai"
suggestions = scraper.suggest_analyze(term, count=20)
print(f"Popular '{term}' searches:")
for suggestion in suggestions:
print(f" {suggestion}")
================================================
FILE: docs/conf.py
================================================
project = 'GPlay Scraper'
copyright = '2025, GPlay Scraper'
author = 'GPlay Scraper'
release = '1.0.5'
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.napoleon',
'sphinx.ext.viewcode',
]
try:
import sphinx_copybutton
extensions.append('sphinx_copybutton')
except ImportError:
pass
templates_path = ['_templates']
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
language = 'en'
html_theme = 'sphinx_book_theme'
html_theme_options = {
"repository_url": "https://github.com/mohammedcha/gplay-scraper",
"repository_branch": "main",
"use_repository_button": True,
"use_issues_button": True,
"use_edit_page_button": False,
"use_download_button": True,
"home_page_in_toc": True,
"show_navbar_depth": 2,
"show_toc_level": 2,
"navigation_with_keys": True,
"collapse_navbar": False,
"logo": {
"text": "GPlay Scraper",
},
"extra_footer": "Built with ❤️ using Sphinx Book Theme
",
"search_bar_text": "Search documentation...",
"icon_links": [
{
"name": "GitHub",
"url": "https://github.com/mohammedcha/gplay-scraper",
"icon": "fa-brands fa-github",
"type": "fontawesome",
},
{
"name": "PyPI",
"url": "https://pypi.org/project/gplay-scraper/",
"icon": "fa-brands fa-python",
"type": "fontawesome",
},
],
}
pygments_style = 'monokai'
pygments_dark_style = 'monokai'
html_title = "GPlay Scraper"
html_static_path = ['_static']
html_logo = "_static/logo.png"
html_favicon = "_static/favicon.png"
if 'sphinx_copybutton' in extensions:
copybutton_prompt_text = r">>> |\.\.\. |\$ |In \[\d*\]: | {2,5}\.\.\.: | {5,8}: "
copybutton_prompt_is_regexp = True
================================================
FILE: docs/configuration.rst
================================================
Configuration
=============
Advanced configuration options for GPlay Scraper.
HTTP Client Selection
---------------------
Choose from 7 HTTP clients with automatic fallback.
.. code-block:: python
from gplay_scraper import GPlayScraper
# Default (requests)
scraper = GPlayScraper()
# Use curl_cffi (best for bypassing blocks)
scraper = GPlayScraper(http_client='curl_cffi')
# Use tls_client (advanced TLS fingerprinting)
scraper = GPlayScraper(http_client='tls_client')
# Use httpx (modern HTTP/2)
scraper = GPlayScraper(http_client='httpx')
Available HTTP Clients
^^^^^^^^^^^^^^^^^^^^^^
1. **requests** - Default, most compatible
2. **curl_cffi** - Best for anti-bot bypass (Chrome 110 impersonation)
3. **tls_client** - Advanced TLS fingerprinting (Chrome 112)
4. **urllib3** - Low-level HTTP with connection pooling
5. **cloudscraper** - Cloudflare bypass
6. **aiohttp** - Async HTTP support
7. **httpx** - Modern HTTP/2 client
The library automatically falls back to the next available client if one fails.
Rate Limiting
-------------
Configure delay between requests to avoid rate limits.
.. code-block:: python
from gplay_scraper import Config
# Set rate limit delay (seconds)
Config.RATE_LIMIT_DELAY = 2.0 # 2 seconds between requests
# Or use default (1.0 second)
Config.RATE_LIMIT_DELAY = 1.0
Language & Region
-----------------
Set default language and country for all requests.
.. code-block:: python
from gplay_scraper import Config
# Set default language
Config.DEFAULT_LANGUAGE = 'es' # Spanish
# Set default country
Config.DEFAULT_COUNTRY = 'mx' # Mexico
Common Language Codes
^^^^^^^^^^^^^^^^^^^^^^
* ``en`` - English
* ``es`` - Spanish
* ``fr`` - French
* ``de`` - German
* ``it`` - Italian
* ``pt`` - Portuguese
* ``ja`` - Japanese
* ``ko`` - Korean
* ``zh`` - Chinese
* ``ru`` - Russian
* ``ar`` - Arabic
* ``hi`` - Hindi
Common Country Codes
^^^^^^^^^^^^^^^^^^^^^
* ``us`` - United States
* ``gb`` - United Kingdom
* ``ca`` - Canada
* ``de`` - Germany
* ``fr`` - France
* ``es`` - Spain
* ``mx`` - Mexico
* ``jp`` - Japan
* ``kr`` - South Korea
* ``cn`` - China
* ``in`` - India
* ``br`` - Brazil
Request Timeout
---------------
Configure HTTP request timeout.
.. code-block:: python
from gplay_scraper import Config
# Set timeout (seconds)
Config.DEFAULT_TIMEOUT = 30 # 30 seconds
# Or use default (10 seconds)
Config.DEFAULT_TIMEOUT = 10
Retry Configuration
-------------------
Configure automatic retry behavior.
.. code-block:: python
from gplay_scraper import Config
# Set number of retries
Config.DEFAULT_RETRY_COUNT = 5 # Try 5 times
# Or use default (3 retries)
Config.DEFAULT_RETRY_COUNT = 3
Image Asset Sizes
-----------------
Configure default image size for all requests.
.. code-block:: python
scraper = GPlayScraper()
# Small images (512px)
app = scraper.app_analyze('com.whatsapp', assets='SMALL')
# Medium images (1024px) - default
app = scraper.app_analyze('com.whatsapp', assets='MEDIUM')
# Large images (2048px)
app = scraper.app_analyze('com.whatsapp', assets='LARGE')
# Original size (maximum)
app = scraper.app_analyze('com.whatsapp', assets='ORIGINAL')
Per-Request Configuration
--------------------------
Override defaults for specific requests.
.. code-block:: python
scraper = GPlayScraper()
# Per-request language
app_es = scraper.app_analyze('com.whatsapp', lang='es')
app_fr = scraper.app_analyze('com.whatsapp', lang='fr')
# Per-request country
app_uk = scraper.app_analyze('com.whatsapp', country='gb')
app_de = scraper.app_analyze('com.whatsapp', country='de')
# Per-request images
app_large = scraper.app_analyze('com.whatsapp', assets='LARGE')
Logging
-------
Configure logging level for debugging.
.. code-block:: python
import logging
# Enable debug logging
logging.basicConfig(level=logging.DEBUG)
# Enable info logging
logging.basicConfig(level=logging.INFO)
# Disable logging
logging.basicConfig(level=logging.ERROR)
Complete Configuration Example
-------------------------------
.. code-block:: python
from gplay_scraper import GPlayScraper, Config
import logging
# Configure library
Config.RATE_LIMIT_DELAY = 2.0
Config.DEFAULT_LANGUAGE = 'en'
Config.DEFAULT_COUNTRY = 'us'
Config.DEFAULT_TIMEOUT = 30
Config.DEFAULT_RETRY_COUNT = 5
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
# Initialize with preferred HTTP client
scraper = GPlayScraper(http_client='curl_cffi')
# Use the scraper
app = scraper.app_analyze('com.whatsapp')
Environment Variables
---------------------
You can also use environment variables for configuration.
.. code-block:: bash
# Set in your shell or .env file
export GPLAY_HTTP_CLIENT=curl_cffi
export GPLAY_RATE_LIMIT=2.0
export GPLAY_LANGUAGE=en
export GPLAY_COUNTRY=us
Best Practices
--------------
1. **Use curl_cffi or tls_client** for better success rates
2. **Set rate limiting** to 2+ seconds for large batch operations
3. **Use field filtering** to reduce data transfer and parsing time
4. **Enable logging** during development, disable in production
5. **Handle exceptions** gracefully for production use
6. **Reuse scraper instance** instead of creating new ones
See Also
--------
* :doc:`error_handling` - Error handling guide
* :doc:`examples` - Practical examples
================================================
FILE: docs/error_handling.rst
================================================
Error Handling
==============
Guide to handling errors and exceptions in GPlay Scraper.
Exception Types
---------------
GPlay Scraper provides 6 custom exception types:
AppNotFoundError
^^^^^^^^^^^^^^^^
Raised when an app, developer, or resource is not found.
.. code-block:: python
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import AppNotFoundError
scraper = GPlayScraper()
try:
app = scraper.app_analyze('invalid.app.id')
except AppNotFoundError as e:
print(f"App not found: {e}")
NetworkError
^^^^^^^^^^^^
Raised when network or HTTP errors occur.
.. code-block:: python
from gplay_scraper.exceptions import NetworkError
try:
app = scraper.app_analyze('com.whatsapp')
except NetworkError as e:
print(f"Network error: {e}")
DataParsingError
^^^^^^^^^^^^^^^^
Raised when JSON parsing or data extraction fails.
.. code-block:: python
from gplay_scraper.exceptions import DataParsingError
try:
app = scraper.app_analyze('com.whatsapp')
except DataParsingError as e:
print(f"Parsing error: {e}")
RateLimitError
^^^^^^^^^^^^^^
Raised when rate limits are exceeded.
.. code-block:: python
from gplay_scraper.exceptions import RateLimitError
try:
# Making too many requests too quickly
for i in range(1000):
app = scraper.app_analyze(f'com.app{i}')
except RateLimitError as e:
print(f"Rate limited: {e}")
InvalidAppIdError
^^^^^^^^^^^^^^^^^
Raised when input validation fails.
.. code-block:: python
from gplay_scraper.exceptions import InvalidAppIdError
try:
app = scraper.app_analyze('') # Empty app ID
except InvalidAppIdError as e:
print(f"Invalid input: {e}")
GPlayScraperError
^^^^^^^^^^^^^^^^^
Base exception for all library errors.
.. code-block:: python
from gplay_scraper.exceptions import GPlayScraperError
try:
app = scraper.app_analyze('com.whatsapp')
except GPlayScraperError as e:
print(f"Library error: {e}")
Comprehensive Error Handling
-----------------------------
Handle all common exceptions.
.. code-block:: python
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import (
AppNotFoundError,
NetworkError,
DataParsingError,
RateLimitError,
InvalidAppIdError,
GPlayScraperError
)
scraper = GPlayScraper()
try:
app = scraper.app_analyze('com.whatsapp')
except InvalidAppIdError as e:
print(f"Invalid app ID: {e}")
except AppNotFoundError as e:
print(f"App not found: {e}")
except NetworkError as e:
print(f"Network error: {e}")
except DataParsingError as e:
print(f"Parsing error: {e}")
except RateLimitError as e:
print(f"Rate limited: {e}")
except GPlayScraperError as e:
print(f"Unknown library error: {e}")
Automatic Retries
-----------------
The library automatically retries failed requests with HTTP client fallback.
.. code-block:: python
from gplay_scraper import Config
# Configure retries
Config.DEFAULT_RETRY_COUNT = 5 # Try 5 times
scraper = GPlayScraper()
app = scraper.app_analyze('com.whatsapp')
# Automatically retries up to 5 times if it fails
# Switches HTTP clients between retries
Graceful Degradation
--------------------
Methods return None or empty lists on failure instead of crashing.
.. code-block:: python
scraper = GPlayScraper()
# Returns None if app not found (after retries)
app = scraper.app_analyze('invalid.app')
if app is None:
print("App not found")
# Returns empty list if search fails
results = scraper.search_analyze('invalid query')
if not results:
print("No results found")
Production Error Handling
--------------------------
Example for production use.
.. code-block:: python
import logging
from gplay_scraper import GPlayScraper, Config
from gplay_scraper.exceptions import GPlayScraperError
# Configure logging
logging.basicConfig(
level=logging.ERROR,
format='%(asctime)s - %(levelname)s - %(message)s',
filename='gplay_scraper.log'
)
logger = logging.getLogger(__name__)
# Configure retries
Config.DEFAULT_RETRY_COUNT = 5
scraper = GPlayScraper(http_client='curl_cffi')
def safe_analyze_app(app_id):
"""Safely analyze an app with error handling."""
try:
return scraper.app_analyze(app_id)
except GPlayScraperError as e:
logger.error(f"Failed to analyze {app_id}: {e}")
return None
# Use in production
app_ids = ['com.app1', 'com.app2', 'com.app3']
results = []
for app_id in app_ids:
app = safe_analyze_app(app_id)
if app:
results.append(app)
print(f"Successfully analyzed {len(results)}/{len(app_ids)} apps")
Batch Processing with Error Handling
-------------------------------------
.. code-block:: python
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import GPlayScraperError
scraper = GPlayScraper()
app_ids = ['com.app1', 'com.app2', 'invalid.app', 'com.app3']
successful = []
failed = []
for app_id in app_ids:
try:
app = scraper.app_analyze(app_id)
if app:
successful.append(app)
except GPlayScraperError as e:
failed.append((app_id, str(e)))
print(f"Successful: {len(successful)}")
print(f"Failed: {len(failed)}")
if failed:
print("\nFailed apps:")
for app_id, error in failed:
print(f" {app_id}: {error}")
Best Practices
--------------
1. **Always handle exceptions** in production code
2. **Use specific exceptions** when possible instead of catching all
3. **Log errors** for debugging and monitoring
4. **Implement retries** for transient failures
5. **Use graceful degradation** - continue processing even if some items fail
6. **Monitor error rates** to detect issues early
See Also
--------
* :doc:`configuration` - Configuration options
* :doc:`examples` - More practical examples
================================================
FILE: docs/examples.rst
================================================
Examples
========
Practical examples of using GPlay Scraper for common tasks.
App Analytics Dashboard
-----------------------
Track key metrics for your app.
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
app = scraper.app_analyze('com.myapp')
print("=== App Analytics Dashboard ===")
print(f"App: {app['title']}")
print(f"Developer: {app['developer']}")
print(f"Rating: {app['score']}/5 ({app['ratings']:,} ratings)")
print(f"\nInstall Metrics:")
print(f" Total Installs: {app['realInstalls']:,}")
print(f" Daily Installs: {app['dailyInstalls']:,}")
print(f" Monthly Installs: {app['monthlyInstalls']:,}")
print(f" App Age: {app['appAgeDays']} days")
print(f"\nRating Distribution:")
hist = app['histogram']
for i, count in enumerate(hist, 1):
print(f" {i}★: {count:,}")
Market Research
---------------
Analyze a market segment.
.. code-block:: python
scraper = GPlayScraper()
# Search for fitness apps
results = scraper.search_analyze('fitness tracker', count=100)
# Filter by rating
high_rated = [app for app in results if app['score'] >= 4.5]
free_apps = [app for app in high_rated if app['free']]
print(f"Total fitness tracker apps: {len(results)}")
print(f"High-rated (4.5+): {len(high_rated)}")
print(f"High-rated & Free: {len(free_apps)}")
print("\nTop 5 Free High-Rated Apps:")
for app in free_apps[:5]:
print(f" {app['title']}: {app['score']}/5")
Competitor Monitoring
---------------------
Track your competitors.
.. code-block:: python
scraper = GPlayScraper()
competitors = ['com.competitor1', 'com.competitor2', 'com.competitor3']
print("Competitor Analysis")
print("-" * 60)
for app_id in competitors:
app = scraper.app_analyze(app_id)
reviews = scraper.reviews_analyze(app_id, count=100, sort='NEWEST')
avg_recent_rating = sum(r['score'] for r in reviews) / len(reviews)
print(f"\n{app['title']}")
print(f" Overall Rating: {app['score']}/5")
print(f" Recent Rating: {avg_recent_rating:.2f}/5")
print(f" Daily Installs: {app['dailyInstalls']:,}")
print(f" Total Installs: {app['realInstalls']:,}")
Review Sentiment Analysis
--------------------------
Analyze user feedback.
.. code-block:: python
scraper = GPlayScraper()
reviews = scraper.reviews_analyze('com.myapp', count=500)
# Categorize by rating
positive = [r for r in reviews if r['score'] >= 4]
neutral = [r for r in reviews if r['score'] == 3]
negative = [r for r in reviews if r['score'] <= 2]
print("Review Sentiment Analysis")
print(f"Total Reviews: {len(reviews)}")
print(f"Positive (4-5★): {len(positive)} ({len(positive)/len(reviews)*100:.1f}%)")
print(f"Neutral (3★): {len(neutral)} ({len(neutral)/len(reviews)*100:.1f}%)")
print(f"Negative (1-2★): {len(negative)} ({len(negative)/len(reviews)*100:.1f}%)")
# Show recent negative reviews
print("\nRecent Negative Reviews:")
for review in negative[:5]:
print(f" {review['userName']}: {review['score']}/5")
print(f" {review['content'][:100]}...")
Top Charts Tracking
-------------------
Monitor top charts positions.
.. code-block:: python
scraper = GPlayScraper()
# Track top free games
top_games = scraper.list_analyze('TOP_FREE', category='GAME', count=50)
# Find your app's position
my_app_id = 'com.mygame'
position = next((i for i, app in enumerate(top_games, 1)
if app['appId'] == my_app_id), None)
if position:
print(f"Your game is ranked #{position} in top free games!")
else:
print("Your game is not in top 50")
# Show top 10
print("\nTop 10 Free Games:")
for i, app in enumerate(top_games[:10], 1):
print(f"{i}. {app['title']} - {app['score']}/5")
Developer Portfolio Overview
-----------------------------
Analyze a developer's entire portfolio.
.. code-block:: python
scraper = GPlayScraper()
apps = scraper.developer_analyze('Google LLC')
# Calculate metrics
avg_rating = sum(app['score'] for app in apps) / len(apps)
free_count = sum(1 for app in apps if app['free'])
high_rated = [app for app in apps if app['score'] >= 4.5]
print(f"Developer: Google LLC")
print(f"Total Apps: {len(apps)}")
print(f"Average Rating: {avg_rating:.2f}/5")
print(f"Free Apps: {free_count}/{len(apps)}")
print(f"High-Rated Apps (4.5+): {len(high_rated)}")
# Best rated apps
sorted_apps = sorted(apps, key=lambda x: x['score'], reverse=True)
print("\nTop 5 Highest Rated:")
for app in sorted_apps[:5]:
print(f" {app['title']}: {app['score']}/5")
Batch Data Collection
----------------------
Collect data for multiple apps efficiently.
.. code-block:: python
import json
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
app_ids = [
'com.whatsapp',
'org.telegram.messenger',
'org.thoughtcrime.securesms',
'com.discord'
]
results = []
for app_id in app_ids:
# Get only the fields you need
fields = ['title', 'developer', 'score', 'realInstalls', 'dailyInstalls']
data = scraper.app_get_fields(app_id, fields)
results.append(data)
# Save to JSON
with open('messaging_apps.json', 'w') as f:
json.dump(results, f, indent=2)
print(f"Collected data for {len(results)} apps")
Multi-Language Content
----------------------
Get localized app information.
.. code-block:: python
scraper = GPlayScraper()
languages = {
'en': 'English',
'es': 'Spanish',
'fr': 'French',
'de': 'German',
'ja': 'Japanese'
}
for lang_code, lang_name in languages.items():
app = scraper.app_analyze('com.whatsapp', lang=lang_code)
print(f"\n{lang_name} ({lang_code}):")
print(f" Title: {app['title']}")
print(f" Summary: {app['summary']}")
Trend Discovery
---------------
Discover trending apps in a category.
.. code-block:: python
scraper = GPlayScraper()
# Get top free apps
top_free = scraper.list_analyze('TOP_FREE', category='PRODUCTIVITY', count=100)
# Filter for new apps (less than 180 days old)
new_apps = [app for app in top_free if 'New' in app.get('description', '')]
# Get apps with high install velocity
trending = []
for app in top_free[:20]:
full_data = scraper.app_analyze(app['appId'])
if full_data['dailyInstalls'] > 10000:
trending.append(full_data)
print("Trending Productivity Apps:")
for app in trending:
print(f" {app['title']}")
print(f" Daily Installs: {app['dailyInstalls']:,}")
print(f" Rating: {app['score']}/5")
See Also
--------
* :doc:`quickstart` - Basic usage guide
* :doc:`api/app` - Complete API reference
* :doc:`configuration` - Configuration options
================================================
FILE: docs/fields.rst
================================================
Field Reference
===============
Complete reference of all 112 fields returned by GPlay Scraper.
App Fields (57 Fields)
-----------------------
Basic Information (5 fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* ``appId`` (string) - Package identifier (e.g., "com.whatsapp")
* ``title`` (string) - App name
* ``summary`` (string) - Short description
* ``description`` (string) - Full description
* ``appUrl`` (string) - Play Store URL
Category (4 fields)
^^^^^^^^^^^^^^^^^^^
* ``genre`` (string) - Primary category
* ``genreId`` (string) - Category ID
* ``categories`` (array) - All categories
* ``available`` (boolean) - Availability status
Release & Updates (3 fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* ``released`` (string) - Release date (e.g., "Oct 18, 2010")
* ``appAgeDays`` (integer) - Days since release (computed)
* ``lastUpdated`` (string) - Last update date
Media (5 fields)
^^^^^^^^^^^^^^^^
* ``icon`` (string) - App icon URL
* ``headerImage`` (string) - Header image URL
* ``screenshots`` (array) - Screenshot URLs
* ``video`` (string or null) - Promotional video URL
* ``videoImage`` (string or null) - Video thumbnail URL
Install Statistics (10 fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* ``installs`` (string) - Install range (e.g., "10,000,000,000+")
* ``minInstalls`` (integer) - Minimum installs
* ``realInstalls`` (integer) - Exact install count
* ``dailyInstalls`` (integer) - Average daily installs (computed)
* ``minDailyInstalls`` (integer) - Min daily installs (computed)
* ``realDailyInstalls`` (integer) - Real daily installs (computed)
* ``monthlyInstalls`` (integer) - Average monthly installs (computed)
* ``minMonthlyInstalls`` (integer) - Min monthly installs (computed)
* ``realMonthlyInstalls`` (integer) - Real monthly installs (computed)
Ratings (4 fields)
^^^^^^^^^^^^^^^^^^
* ``score`` (float) - Average rating (0-5)
* ``ratings`` (integer) - Total ratings count
* ``reviews`` (integer) - Total reviews count
* ``histogram`` (array) - Rating distribution [1★, 2★, 3★, 4★, 5★]
Ads (2 fields)
^^^^^^^^^^^^^^
* ``adSupported`` (boolean) - Supports ads
* ``containsAds`` (boolean) - Contains ads
Technical (7 fields)
^^^^^^^^^^^^^^^^^^^^
* ``version`` (string) - Current version
* ``androidVersion`` (string) - Minimum Android version
* ``maxAndroidApi`` (integer) - Maximum Android API
* ``minAndroidApi`` (string or integer) - Minimum Android API
* ``appBundle`` (string) - Bundle identifier
* ``contentRating`` (string) - Age rating
* ``contentRatingDescription`` (string) - Rating description
Updates (1 field)
^^^^^^^^^^^^^^^^^
* ``whatsNew`` (array) - Changelog entries
Privacy (2 fields)
^^^^^^^^^^^^^^^^^^
* ``permissions`` (object) - Required permissions
* ``dataSafety`` (array) - Data safety information
Pricing (7 fields)
^^^^^^^^^^^^^^^^^^
* ``price`` (number) - App price
* ``currency`` (string) - Currency code
* ``free`` (boolean) - Is free
* ``offersIAP`` (boolean) - Has in-app purchases
* ``inAppProductPrice`` (string or null) - IAP price range
* ``sale`` (boolean) - On sale
* ``originalPrice`` (number or null) - Original price if on sale
Developer (8 fields)
^^^^^^^^^^^^^^^^^^^^
* ``developer`` (string) - Developer name
* ``developerId`` (string) - Developer ID
* ``developerEmail`` (string) - Contact email
* ``developerWebsite`` (string) - Website URL
* ``developerAddress`` (string) - Physical address
* ``developerPhone`` (string or null) - Contact phone
* ``publisherCountry`` (string) - Publisher country (computed)
* ``privacyPolicy`` (string) - Privacy policy URL
Search Fields (11 Fields)
--------------------------
* ``title`` - App name
* ``appId`` - Package identifier
* ``url`` - Play Store URL
* ``icon`` - App icon URL
* ``developer`` - Developer name
* ``summary`` - Short description
* ``score`` - Average rating (0-5)
* ``scoreText`` - Rating as text
* ``price`` - App price
* ``free`` - Is free (boolean)
* ``currency`` - Currency code
Review Fields (8 Fields)
-------------------------
* ``reviewId`` - Unique review ID
* ``userName`` - Reviewer name
* ``userImage`` - Reviewer image URL
* ``content`` - Review text
* ``score`` - Rating (1-5)
* ``thumbsUpCount`` - Helpful votes
* ``at`` - Review date (ISO format)
* ``appVersion`` - App version reviewed
Developer App Fields (11 Fields)
---------------------------------
Same as Search Fields.
Similar App Fields (11 Fields)
-------------------------------
Same as Search Fields.
List (Top Charts) Fields (14 Fields)
-------------------------------------
* ``title`` - App name
* ``appId`` - Package identifier
* ``url`` - Play Store URL
* ``icon`` - App icon URL
* ``screenshots`` - Screenshot URLs (array)
* ``developer`` - Developer name
* ``genre`` - Category/genre
* ``installs`` - Install count
* ``description`` - App description
* ``score`` - Average rating
* ``scoreText`` - Rating as text
* ``price`` - App price
* ``free`` - Is free (boolean)
* ``currency`` - Currency code
Computed Fields
---------------
The following 8 fields are computed at runtime:
**appAgeDays**
Calculated as: ``(current_date - release_date).days``
**dailyInstalls**
Calculated as: ``total_installs / days_since_release``
**minDailyInstalls**
Calculated as: ``min_installs / days_since_release``
**realDailyInstalls**
Calculated as: ``real_installs / days_since_release``
**monthlyInstalls**
Calculated as: ``total_installs / (days_since_release / 30.44)``
**minMonthlyInstalls**
Calculated as: ``min_installs / months_since_release``
**realMonthlyInstalls**
Calculated as: ``real_installs / months_since_release``
**publisherCountry**
Extracted from developer phone prefix or address
Field Count Summary
-------------------
* App: 57 fields
* Search: 11 fields
* Reviews: 8 fields
* Developer: 11 fields
* Similar: 11 fields
* List: 14 fields
* **Total: 112 unique fields**
================================================
FILE: docs/index.rst
================================================
GPlay Scraper Documentation
============================
A comprehensive Python library for scraping Google Play Store data with 40 methods across 7 categories.
Features
--------
* **57 app fields** including install analytics
* **40 methods** for different data types
* **7 HTTP clients** with automatic fallback
* **Multi-language** and **multi-region** support
* **Automatic retries** and error handling
* **Rate limiting** built-in
Quick Example
-------------
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
# Get complete app data (57 fields)
app = scraper.app_analyze('com.whatsapp')
print(app['title']) # WhatsApp Messenger
print(app['realInstalls']) # 10931553905
print(app['dailyInstalls']) # 1815870
print(app['publisherCountry']) # United States
Table of Contents
-----------------
.. toctree::
:maxdepth: 2
:caption: Getting Started
installation
quickstart
examples
.. toctree::
:maxdepth: 2
:caption: API Reference
api/app
api/search
api/reviews
api/developer
api/similar
api/list
api/suggest
.. toctree::
:maxdepth: 2
:caption: Advanced
configuration
error_handling
fields
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
================================================
FILE: docs/installation.rst
================================================
Installation
============
Requirements
------------
* Python 3.7 or higher
* pip package manager
Basic Installation
------------------
Install using pip:
.. code-block:: bash
pip install gplay-scraper
This installs the library with the default HTTP client (requests).
Optional Dependencies
---------------------
For better performance and anti-bot protection, install additional HTTP clients:
.. code-block:: bash
# Install all optional HTTP clients
pip install httpx curl-cffi tls-client aiohttp cloudscraper
# Or install individually as needed
pip install httpx # Modern HTTP/2 client
pip install curl-cffi # Best for bypassing blocks
pip install tls-client # Advanced TLS fingerprinting
pip install aiohttp # Async support
pip install cloudscraper # Cloudflare bypass
Verify Installation
-------------------
Test your installation:
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
app = scraper.app_analyze('com.whatsapp')
print(f"Successfully installed! Got: {app['title']}")
Development Installation
------------------------
To install from source:
.. code-block:: bash
git clone https://github.com/yourusername/gplay-scraper.git
cd gplay-scraper
pip install -e .
Upgrading
---------
To upgrade to the latest version:
.. code-block:: bash
pip install --upgrade gplay-scraper
Troubleshooting
---------------
ImportError
^^^^^^^^^^^
If you get an ImportError, ensure the package is installed:
.. code-block:: bash
pip show gplay-scraper
HTTP Client Issues
^^^^^^^^^^^^^^^^^^
If you encounter HTTP errors, try installing alternative clients:
.. code-block:: bash
pip install curl-cffi
Then specify the client:
.. code-block:: python
from gplay_scraper import GPlayScraper
scraper = GPlayScraper(http_client='curl_cffi')
Next Steps
----------
* :doc:`quickstart` - Get started with basic usage
* :doc:`examples` - See practical examples
* :doc:`api/app` - Explore the API reference
================================================
FILE: docs/quickstart.rst
================================================
Quick Start Guide
=================
This guide will get you started with GPlay Scraper in 5 minutes.
Basic Usage
-----------
Initialize the Scraper
^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
from gplay_scraper import GPlayScraper
# Initialize once
scraper = GPlayScraper()
Get App Data
^^^^^^^^^^^^
Extract complete app information with 57 fields:
.. code-block:: python
# Get all app data
app = scraper.app_analyze('com.whatsapp')
# Access the data
print(app['title']) # App name
print(app['developer']) # Developer name
print(app['score']) # Rating (0-5)
print(app['realInstalls']) # Exact install count
print(app['dailyInstalls']) # Average daily installs
print(app['publisherCountry']) # Publisher country
Get Specific Fields
^^^^^^^^^^^^^^^^^^^
If you only need certain fields:
.. code-block:: python
# Get single field
title = scraper.app_get_field('com.whatsapp', 'title')
# Get multiple fields
fields = scraper.app_get_fields('com.whatsapp',
['title', 'score', 'dailyInstalls'])
Search for Apps
^^^^^^^^^^^^^^^
Search the Play Store by keyword:
.. code-block:: python
# Search for apps
results = scraper.search_analyze('messaging', count=10)
# Iterate through results
for app in results:
print(f"{app['title']} by {app['developer']}")
print(f" Rating: {app['score']}/5")
print(f" Free: {app['free']}")
Get Reviews
^^^^^^^^^^^
Extract user reviews with ratings:
.. code-block:: python
# Get newest reviews
reviews = scraper.reviews_analyze('com.whatsapp',
count=50,
sort='NEWEST')
# Process reviews
for review in reviews:
print(f"{review['userName']}: {review['score']}/5")
print(f" {review['content'][:100]}...")
Get Developer Apps
^^^^^^^^^^^^^^^^^^
Find all apps from a developer:
.. code-block:: python
# Get all apps from Google
apps = scraper.developer_analyze('Google LLC')
for app in apps:
print(f"{app['title']} - {app['score']}/5")
Find Similar Apps
^^^^^^^^^^^^^^^^^
Discover competitor or similar apps:
.. code-block:: python
# Find apps similar to WhatsApp
similar = scraper.similar_analyze('com.whatsapp', count=20)
for app in similar:
print(f"{app['title']} - {app['score']}/5")
Get Top Charts
^^^^^^^^^^^^^^
Access top free, paid, or grossing apps:
.. code-block:: python
# Top free games
top_free = scraper.list_analyze('TOP_FREE',
category='GAME',
count=100)
# Top paid apps
top_paid = scraper.list_analyze('TOP_PAID',
category='APPLICATION',
count=50)
# Top grossing
top_grossing = scraper.list_analyze('TOP_GROSSING', count=100)
Get Search Suggestions
^^^^^^^^^^^^^^^^^^^^^^
Get autocomplete suggestions:
.. code-block:: python
# Get suggestions
suggestions = scraper.suggest_analyze('mine', count=10)
print(suggestions)
# ['minecraft', 'minesweeper', 'mineplex', ...]
Multi-Language Support
----------------------
Get data in different languages:
.. code-block:: python
# Spanish
app = scraper.app_analyze('com.whatsapp', lang='es')
# French
app = scraper.app_analyze('com.whatsapp', lang='fr')
# Japanese
app = scraper.app_analyze('com.whatsapp', lang='ja')
Regional Data
-------------
Get region-specific data:
.. code-block:: python
# UK data
app = scraper.app_analyze('com.whatsapp', country='gb')
# Germany
app = scraper.app_analyze('com.whatsapp', country='de')
# Japan
app = scraper.app_analyze('com.whatsapp', country='jp')
Image Sizes
-----------
Control image quality:
.. code-block:: python
# Small images (512px)
app = scraper.app_analyze('com.whatsapp', assets='SMALL')
# Medium images (1024px) - default
app = scraper.app_analyze('com.whatsapp', assets='MEDIUM')
# Large images (2048px)
app = scraper.app_analyze('com.whatsapp', assets='LARGE')
# Original size
app = scraper.app_analyze('com.whatsapp', assets='ORIGINAL')
Error Handling
--------------
Handle errors gracefully:
.. code-block:: python
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import (
AppNotFoundError,
InvalidAppIdError,
NetworkError
)
scraper = GPlayScraper()
try:
app = scraper.app_analyze('invalid.app.id')
except InvalidAppIdError:
print("Invalid app ID format")
except AppNotFoundError:
print("App not found on Play Store")
except NetworkError:
print("Network error occurred")
Common Patterns
---------------
Batch Processing
^^^^^^^^^^^^^^^^
.. code-block:: python
app_ids = ['com.whatsapp', 'com.telegram', 'com.signal']
for app_id in app_ids:
app = scraper.app_analyze(app_id)
print(f"{app['title']}: {app['realInstalls']:,} installs")
Market Research
^^^^^^^^^^^^^^^
.. code-block:: python
# Find highly-rated messaging apps
results = scraper.search_analyze('messaging', count=100)
high_rated = [app for app in results if app['score'] >= 4.5]
for app in high_rated:
print(f"{app['title']}: {app['score']}/5")
Competitor Analysis
^^^^^^^^^^^^^^^^^^^
.. code-block:: python
# Analyze your app vs competitors
my_app = scraper.app_analyze('com.myapp')
competitors = scraper.similar_analyze('com.myapp', count=10)
print(f"My App: {my_app['score']}/5")
print("\nCompetitors:")
for comp in competitors:
print(f" {comp['title']}: {comp['score']}/5")
Review Monitoring
^^^^^^^^^^^^^^^^^
.. code-block:: python
# Monitor negative reviews
reviews = scraper.reviews_analyze('com.myapp',
count=100,
sort='NEWEST')
negative = [r for r in reviews if r['score'] <= 2]
for review in negative:
print(f"{review['userName']}: {review['score']}/5")
print(f" {review['content']}")
Next Steps
----------
* :doc:`examples` - See more detailed examples
* :doc:`api/app` - Complete API reference
* :doc:`configuration` - Advanced configuration options
* :doc:`fields` - All available fields reference
================================================
FILE: docs/requirements.txt
================================================
sphinx>=7.0.0
sphinx-book-theme>=1.0.0
sphinx-copybutton>=0.5.0
sphinx-autobuild>=2021.3.14
================================================
FILE: examples/README.md
================================================
# Examples
This folder contains example scripts demonstrating all methods for each of the 7 method types.
## Files
### 1. app_methods_example.py
Demonstrates all 6 app methods:
- `app_analyze()` - Get all data as dictionary
- `app_get_field()` - Get single field value
- `app_get_fields()` - Get multiple fields
- `app_print_field()` - Print single field to console
- `app_print_fields()` - Print multiple fields to console
- `app_print_all()` - Print all data as JSON
### 2. search_methods_example.py
Demonstrates all 6 search methods:
- `search_analyze()` - Get all search results
- `search_get_field()` - Get single field from results
- `search_get_fields()` - Get multiple fields from results
- `search_print_field()` - Print single field from results
- `search_print_fields()` - Print multiple fields from results
- `search_print_all()` - Print all results as JSON
### 3. reviews_methods_example.py
Demonstrates all 6 reviews methods:
- `reviews_analyze()` - Get all reviews
- `reviews_get_field()` - Get single field from reviews
- `reviews_get_fields()` - Get multiple fields from reviews
- `reviews_print_field()` - Print single field from reviews
- `reviews_print_fields()` - Print multiple fields from reviews
- `reviews_print_all()` - Print all reviews as JSON
### 4. developer_methods_example.py
Demonstrates all 6 developer methods:
- `developer_analyze()` - Get all developer apps
- `developer_get_field()` - Get single field from apps
- `developer_get_fields()` - Get multiple fields from apps
- `developer_print_field()` - Print single field from apps
- `developer_print_fields()` - Print multiple fields from apps
- `developer_print_all()` - Print all apps as JSON
### 5. list_methods_example.py
Demonstrates all 6 list methods:
- `list_analyze()` - Get all top chart apps
- `list_get_field()` - Get single field from apps
- `list_get_fields()` - Get multiple fields from apps
- `list_print_field()` - Print single field from apps
- `list_print_fields()` - Print multiple fields from apps
- `list_print_all()` - Print all apps as JSON
### 6. similar_methods_example.py
Demonstrates all 6 similar methods:
- `similar_analyze()` - Get all similar apps
- `similar_get_field()` - Get single field from apps
- `similar_get_fields()` - Get multiple fields from apps
- `similar_print_field()` - Print single field from apps
- `similar_print_fields()` - Print multiple fields from apps
- `similar_print_all()` - Print all apps as JSON
### 7. suggest_methods_example.py
Demonstrates all 4 suggest methods:
- `suggest_analyze()` - Get search suggestions
- `suggest_nested()` - Get nested suggestions
- `suggest_print_all()` - Print suggestions as JSON
- `suggest_print_nested()` - Print nested suggestions as JSON
## Running Examples
```bash
# Run any example
python examples/app_methods_example.py
python examples/search_methods_example.py
python examples/reviews_methods_example.py
python examples/developer_methods_example.py
python examples/list_methods_example.py
python examples/similar_methods_example.py
python examples/suggest_methods_example.py
```
## Note
These examples are simple demonstrations. For more advanced use cases, check the documentation in the `README/` folder.
================================================
FILE: examples/app_methods_example.py
================================================
"""
App Methods Example
Demonstrates all 6 app methods for extracting app details
Parameters:
- app_id: App package name
- lang: Language code (default: 'en')
- country: Country code (default: 'us')
"""
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
app_id = "com.whatsapp"
lang = "en"
country = "us"
print("=== App Methods Example ===\n")
# 1. app_analyze() - Get all data as dictionary
print("1. app_analyze(app_id, lang='en', country='us')")
data = scraper.app_analyze(app_id, lang=lang, country=country)
print(f" Retrieved {len(data)} fields")
print(f" Title: {data['title']}")
print(f" Score: {data['score']}")
# 2. app_get_field() - Get single field
print("\n2. app_get_field(app_id, field, lang='en', country='us')")
title = scraper.app_get_field(app_id, "title", lang=lang, country=country)
print(f" Title: {title}")
# 3. app_get_fields() - Get multiple fields
print("\n3. app_get_fields(app_id, fields, lang='en', country='us')")
fields = scraper.app_get_fields(app_id, ["title", "score", "installs"], lang=lang, country=country)
print(f" {fields}")
# 4. app_print_field() - Print single field
print("\n4. app_print_field(app_id, field, lang='en', country='us')")
scraper.app_print_field(app_id, "developer", lang=lang, country=country)
# 5. app_print_fields() - Print multiple fields
print("\n5. app_print_fields(app_id, fields, lang='en', country='us')")
scraper.app_print_fields(app_id, ["title", "score", "free"], lang=lang, country=country)
# 6. app_print_all() - Print all data as JSON
print("\n6. app_print_all(app_id, lang='en', country='us')")
scraper.app_print_all(app_id, lang=lang, country=country)
================================================
FILE: examples/developer_methods_example.py
================================================
"""
Developer Methods Example
Demonstrates all 6 developer methods for getting developer's apps
Parameters:
- dev_id: Developer ID (numeric or string)
- count: Number of apps (default: 100)
- lang: Language code (default: 'en')
- country: Country code (default: 'us')
"""
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
dev_id = "5700313618786177705" # Google LLC
count = 20
lang = "en"
country = "us"
print("=== Developer Methods Example ===\n")
# 1. developer_analyze() - Get all developer apps
print("1. developer_analyze(dev_id, count=100, lang='en', country='us')")
apps = scraper.developer_analyze(dev_id, count=count, lang=lang, country=country)
print(f" Found {len(apps)} apps")
print(f" First app: {apps[0]['title']}")
# 2. developer_get_field() - Get single field from all apps
print("\n2. developer_get_field(dev_id, field, count=100, lang='en', country='us')")
titles = scraper.developer_get_field(dev_id, "title", count=count, lang=lang, country=country)
print(f" Titles: {titles[:3]}")
# 3. developer_get_fields() - Get multiple fields from all apps
print("\n3. developer_get_fields(dev_id, fields, count=100, lang='en', country='us')")
apps_data = scraper.developer_get_fields(dev_id, ["title", "score"], count=10, lang=lang, country=country)
print(f" First 2 apps: {apps_data[:2]}")
# 4. developer_print_field() - Print single field from all apps
print("\n4. developer_print_field(dev_id, field, count=100, lang='en', country='us')")
scraper.developer_print_field(dev_id, "title", count=5, lang=lang, country=country)
# 5. developer_print_fields() - Print multiple fields from all apps
print("\n5. developer_print_fields(dev_id, fields, count=100, lang='en', country='us')")
scraper.developer_print_fields(dev_id, ["title", "score"], count=5, lang=lang, country=country)
# 6. developer_print_all() - Print all developer apps as JSON
print("\n6. developer_print_all(dev_id, count=100, lang='en', country='us')")
scraper.developer_print_all(dev_id, count=5, lang=lang, country=country)
================================================
FILE: examples/list_methods_example.py
================================================
"""
List Methods Example
Demonstrates all 6 list methods for getting top charts
Parameters:
- collection: Chart type - 'TOP_FREE', 'TOP_PAID', 'TOP_GROSSING' (default: 'TOP_FREE')
- category: Category filter (default: 'APPLICATION')
- count: Number of apps (default: 100)
- lang: Language code (default: 'en')
- country: Country code (default: 'us')
"""
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
collection = "TOP_FREE"
category = "GAME"
count = 20
lang = "en"
country = "us"
print("=== List Methods Example ===\n")
# 1. list_analyze() - Get all top chart apps
print("1. list_analyze(collection='TOP_FREE', category='APPLICATION', count=100, lang='en', country='us')")
apps = scraper.list_analyze(collection, category, count=count, lang=lang, country=country)
print(f" Found {len(apps)} apps")
print(f" First app: {apps[0]['title']}")
# 2. list_get_field() - Get single field from all apps
print("\n2. list_get_field(collection, field, category='APPLICATION', count=100, lang='en', country='us')")
titles = scraper.list_get_field(collection, "title", category, count=count, lang=lang, country=country)
print(f" Titles: {titles[:3]}")
# 3. list_get_fields() - Get multiple fields from all apps
print("\n3. list_get_fields(collection, fields, category='APPLICATION', count=100, lang='en', country='us')")
apps_data = scraper.list_get_fields(collection, ["title", "score"], category, count=10, lang=lang, country=country)
print(f" First 2 apps: {apps_data[:2]}")
# 4. list_print_field() - Print single field from all apps
print("\n4. list_print_field(collection, field, category='APPLICATION', count=100, lang='en', country='us')")
scraper.list_print_field(collection, "title", category, count=5, lang=lang, country=country)
# 5. list_print_fields() - Print multiple fields from all apps
print("\n5. list_print_fields(collection, fields, category='APPLICATION', count=100, lang='en', country='us')")
scraper.list_print_fields(collection, ["title", "score"], category, count=5, lang=lang, country=country)
# 6. list_print_all() - Print all top chart apps as JSON
print("\n6. list_print_all(collection='TOP_FREE', category='APPLICATION', count=100, lang='en', country='us')")
scraper.list_print_all(collection, category, count=5, lang=lang, country=country)
================================================
FILE: examples/reviews_methods_example.py
================================================
"""
Reviews Methods Example
Demonstrates all 6 reviews methods for extracting user reviews
Parameters:
- app_id: App package name
- count: Number of reviews (default: 100)
- lang: Language code (default: 'en')
- country: Country code (default: 'us')
- sort: Sort order - 'NEWEST', 'RELEVANT', 'RATING' (default: 'NEWEST')
"""
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
app_id = "com.whatsapp"
count = 20
lang = "en"
country = "us"
sort = "NEWEST"
print("=== Reviews Methods Example ===\n")
# 1. reviews_analyze() - Get all reviews
print("1. reviews_analyze(app_id, count=100, lang='en', country='us', sort='NEWEST')")
reviews = scraper.reviews_analyze(app_id, count=count, lang=lang, country=country, sort=sort)
print(f" Retrieved {len(reviews)} reviews")
print(f" First review score: {reviews[0]['score']}")
# 2. reviews_get_field() - Get single field from all reviews
print("\n2. reviews_get_field(app_id, field, count=100, lang='en', country='us', sort='NEWEST')")
scores = scraper.reviews_get_field(app_id, "score", count=count, lang=lang, country=country, sort=sort)
print(f" Scores: {scores[:5]}")
# 3. reviews_get_fields() - Get multiple fields from all reviews
print("\n3. reviews_get_fields(app_id, fields, count=100, lang='en', country='us', sort='NEWEST')")
review_data = scraper.reviews_get_fields(app_id, ["userName", "score"], count=10, lang=lang, country=country, sort=sort)
print(f" First 2 reviews: {review_data[:2]}")
# 4. reviews_print_field() - Print single field from all reviews
print("\n4. reviews_print_field(app_id, field, count=100, lang='en', country='us', sort='NEWEST')")
scraper.reviews_print_field(app_id, "score", count=5, lang=lang, country=country, sort=sort)
# 5. reviews_print_fields() - Print multiple fields from all reviews
print("\n5. reviews_print_fields(app_id, fields, count=100, lang='en', country='us', sort='NEWEST')")
scraper.reviews_print_fields(app_id, ["userName", "score"], count=5, lang=lang, country=country, sort=sort)
# 6. reviews_print_all() - Print all reviews as JSON
print("\n6. reviews_print_all(app_id, count=100, lang='en', country='us', sort='NEWEST')")
scraper.reviews_print_all(app_id, count=5, lang=lang, country=country, sort=sort)
================================================
FILE: examples/search_methods_example.py
================================================
"""
Search Methods Example
Demonstrates all 6 search methods for finding apps
Parameters:
- query: Search keyword
- count: Number of results (default: 100)
- lang: Language code (default: 'en')
- country: Country code (default: 'us')
"""
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
query = "social media"
count = 10
lang = "en"
country = "us"
print("=== Search Methods Example ===\n")
# 1. search_analyze() - Get all search results
print("1. search_analyze(query, count=100, lang='en', country='us')")
results = scraper.search_analyze(query, count=count, lang=lang, country=country)
print(f" Found {len(results)} apps")
print(f" First app: {results[0]['title']}")
# 2. search_get_field() - Get single field from all results
print("\n2. search_get_field(query, field, count=100, lang='en', country='us')")
titles = scraper.search_get_field(query, "title", count=count, lang=lang, country=country)
print(f" Titles: {titles[:3]}")
# 3. search_get_fields() - Get multiple fields from all results
print("\n3. search_get_fields(query, fields, count=100, lang='en', country='us')")
apps = scraper.search_get_fields(query, ["title", "score"], count=count, lang=lang, country=country)
print(f" First 2 apps: {apps[:2]}")
# 4. search_print_field() - Print single field from all results
print("\n4. search_print_field(query, field, count=100, lang='en', country='us')")
scraper.search_print_field(query, "title", count=5, lang=lang, country=country)
# 5. search_print_fields() - Print multiple fields from all results
print("\n5. search_print_fields(query, fields, count=100, lang='en', country='us')")
scraper.search_print_fields(query, ["title", "developer"], count=5, lang=lang, country=country)
# 6. search_print_all() - Print all search results as JSON
print("\n6. search_print_all(query, count=100, lang='en', country='us')")
scraper.search_print_all(query, count=5, lang=lang, country=country)
================================================
FILE: examples/similar_methods_example.py
================================================
"""
Similar Methods Example
Demonstrates all 6 similar methods for finding related apps
Parameters:
- app_id: App package name
- count: Number of similar apps (default: 100)
- lang: Language code (default: 'en')
- country: Country code (default: 'us')
"""
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
app_id = "com.whatsapp"
count = 20
lang = "en"
country = "us"
print("=== Similar Methods Example ===\n")
# 1. similar_analyze() - Get all similar apps
print("1. similar_analyze(app_id, count=100, lang='en', country='us')")
apps = scraper.similar_analyze(app_id, count=count, lang=lang, country=country)
print(f" Found {len(apps)} similar apps")
print(f" First app: {apps[0]['title']}")
# 2. similar_get_field() - Get single field from all similar apps
print("\n2. similar_get_field(app_id, field, count=100, lang='en', country='us')")
titles = scraper.similar_get_field(app_id, "title", count=count, lang=lang, country=country)
print(f" Titles: {titles[:3]}")
# 3. similar_get_fields() - Get multiple fields from all similar apps
print("\n3. similar_get_fields(app_id, fields, count=100, lang='en', country='us')")
apps_data = scraper.similar_get_fields(app_id, ["title", "score"], count=10, lang=lang, country=country)
print(f" First 2 apps: {apps_data[:2]}")
# 4. similar_print_field() - Print single field from all similar apps
print("\n4. similar_print_field(app_id, field, count=100, lang='en', country='us')")
scraper.similar_print_field(app_id, "title", count=5, lang=lang, country=country)
# 5. similar_print_fields() - Print multiple fields from all similar apps
print("\n5. similar_print_fields(app_id, fields, count=100, lang='en', country='us')")
scraper.similar_print_fields(app_id, ["title", "score"], count=5, lang=lang, country=country)
# 6. similar_print_all() - Print all similar apps as JSON
print("\n6. similar_print_all(app_id, count=100, lang='en', country='us')")
scraper.similar_print_all(app_id, count=5, lang=lang, country=country)
================================================
FILE: examples/suggest_methods_example.py
================================================
"""
Suggest Methods Example
Demonstrates all 4 suggest methods for getting search suggestions
Parameters:
- term: Search term
- count: Number of suggestions (default: 5)
- lang: Language code (default: 'en')
- country: Country code (default: 'us')
"""
from gplay_scraper import GPlayScraper
scraper = GPlayScraper()
term = "fitness"
count = 5
lang = "en"
country = "us"
print("=== Suggest Methods Example ===\n")
# 1. suggest_analyze() - Get search suggestions
print("1. suggest_analyze(term, count=5, lang='en', country='us')")
suggestions = scraper.suggest_analyze(term, count=count, lang=lang, country=country)
print(f" Suggestions: {suggestions}")
# 2. suggest_nested() - Get nested suggestions
print("\n2. suggest_nested(term, count=5, lang='en', country='us')")
nested = scraper.suggest_nested(term, count=count, lang=lang, country=country)
print(f" Nested suggestions (first 2):")
for i, (key, values) in enumerate(list(nested.items())[:2]):
print(f" {key}: {values}")
# 3. suggest_print_all() - Print suggestions as JSON
print("\n3. suggest_print_all(term, count=5, lang='en', country='us')")
scraper.suggest_print_all(term, count=count, lang=lang, country=country)
# 4. suggest_print_nested() - Print nested suggestions as JSON
print("\n4. suggest_print_nested(term, count=5, lang='en', country='us')")
scraper.suggest_print_nested(term, count=count, lang=lang, country=country)
================================================
FILE: gplay_scraper/__init__.py
================================================
"""GPlay Scraper - Google Play Store scraping library.
This package provides comprehensive tools for scraping Google Play Store data including:
- App details (65+ fields)
- Search results
- User reviews
- Developer portfolios
- Similar apps
- Top charts
- Search suggestions
"""
import logging
# Import main scraper class
from .app import GPlayScraper
# Import all method classes
from .core.gplay_methods import AppMethods, SearchMethods, ReviewsMethods, DeveloperMethods, SimilarMethods, ListMethods, SuggestMethods
# Import configuration
from .config import Config
# Import custom exceptions
from .exceptions import (
GPlayScraperError,
InvalidAppIdError,
AppNotFoundError,
RateLimitError,
NetworkError,
DataParsingError,
)
# Configure logging to use NullHandler by default
logging.getLogger(__name__).addHandler(logging.NullHandler())
# Package metadata
__version__ = "1.0.6"
# Public API exports
__all__ = [
"GPlayScraper",
"AppMethods",
"SearchMethods",
"ReviewsMethods",
"DeveloperMethods",
"SimilarMethods",
"ListMethods",
"SuggestMethods",
"Config",
"GPlayScraperError",
"InvalidAppIdError",
"AppNotFoundError",
"RateLimitError",
"NetworkError",
"DataParsingError",
]
================================================
FILE: gplay_scraper/app.py
================================================
"""Main GPlayScraper class that provides unified access to all scraping methods.
This module contains the main GPlayScraper class which aggregates all 7 method types
and provides 42 functions for interacting with Google Play Store data.
"""
from .core.gplay_methods import AppMethods, SearchMethods, ReviewsMethods, DeveloperMethods, SimilarMethods, ListMethods, SuggestMethods
from .config import Config
from typing import Any, List, Dict
class GPlayScraper:
"""Main scraper class providing access to all Google Play Store scraping methods.
This class aggregates 7 method types:
- App Methods: Extract 65+ fields from any app
- Search Methods: Search for apps by keyword
- Reviews Methods: Extract user reviews and ratings
- Developer Methods: Get all apps from a developer
- List Methods: Get top charts (free, paid, grossing)
- Similar Methods: Find similar/competitor apps
- Suggest Methods: Get search suggestions
Args:
http_client: HTTP client to use (requests, curl_cffi, tls_client, httpx, urllib3, cloudscraper, aiohttp)
"""
def __init__(self, http_client: str = None):
"""Initialize GPlayScraper with all method types.
Args:
http_client: Optional HTTP client name. Defaults to 'requests' with automatic fallback.
"""
# Initialize all 7 method types
self.app_methods = AppMethods(http_client)
self.search_methods = SearchMethods(http_client)
self.reviews_methods = ReviewsMethods(http_client)
self.developer_methods = DeveloperMethods(http_client)
self.similar_methods = SimilarMethods(http_client)
self.list_methods = ListMethods(http_client)
self.suggest_methods = SuggestMethods(http_client)
# ==================== App Methods ====================
def app_analyze(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> Dict:
"""Get complete app data with 65+ fields.
Args:
app_id: Google Play app ID (e.g., 'com.whatsapp')
lang: Language code (default: 'en')
country: Country code (default: 'us')
assets: Asset size (SMALL=512px, MEDIUM=1024px, LARGE=2048px, ORIGINAL=max)
Returns:
Dictionary containing all app data
"""
return self.app_methods.app_analyze(app_id, lang, country, assets)
def app_get_field(self, app_id: str, field: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> Any:
"""Get single field value from app data.
Args:
app_id: Google Play app ID
field: Field name to retrieve
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
Returns:
Value of the requested field
"""
return self.app_methods.app_get_field(app_id, field, lang, country, assets)
def app_get_fields(self, app_id: str, fields: List[str], lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> Dict[str, Any]:
"""Get multiple field values from app data.
Args:
app_id: Google Play app ID
fields: List of field names to retrieve
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
Returns:
Dictionary with requested fields and values
"""
return self.app_methods.app_get_fields(app_id, fields, lang, country, assets)
def app_print_field(self, app_id: str, field: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> None:
"""Print single field value to console.
Args:
app_id: Google Play app ID
field: Field name to print
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
"""
return self.app_methods.app_print_field(app_id, field, lang, country, assets)
def app_print_fields(self, app_id: str, fields: List[str], lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> None:
"""Print multiple field values to console.
Args:
app_id: Google Play app ID
fields: List of field names to print
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
"""
return self.app_methods.app_print_fields(app_id, fields, lang, country, assets)
def app_print_all(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> None:
"""Print all app data as JSON to console.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
"""
return self.app_methods.app_print_all(app_id, lang, country, assets)
# ==================== Search Methods ====================
def search_analyze(self, query: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict]:
"""Search for apps and get complete results.
Args:
query: Search query string
count: Number of results to return
lang: Language code
country: Country code
Returns:
List of dictionaries containing app data
"""
return self.search_methods.search_analyze(query, count, lang, country)
def search_get_field(self, query: str, field: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Any]:
"""Get single field from search results.
Args:
query: Search query string
field: Field name to retrieve
count: Number of results
lang: Language code
country: Country code
Returns:
List of field values
"""
return self.search_methods.search_get_field(query, field, count, lang, country)
def search_get_fields(self, query: str, fields: List[str], count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict[str, Any]]:
"""Get multiple fields from search results.
Args:
query: Search query string
fields: List of field names
count: Number of results
lang: Language code
country: Country code
Returns:
List of dictionaries with requested fields
"""
return self.search_methods.search_get_fields(query, fields, count, lang, country)
def search_print_field(self, query: str, field: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print single field from search results.
Args:
query: Search query string
field: Field name to print
count: Number of results
lang: Language code
country: Country code
"""
return self.search_methods.search_print_field(query, field, count, lang, country)
def search_print_fields(self, query: str, fields: List[str], count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print multiple fields from search results.
Args:
query: Search query string
fields: List of field names
count: Number of results
lang: Language code
country: Country code
"""
return self.search_methods.search_print_fields(query, fields, count, lang, country)
def search_print_all(self, query: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all search results as JSON.
Args:
query: Search query string
count: Number of results
lang: Language code
country: Country code
"""
return self.search_methods.search_print_all(query, count, lang, country)
# ==================== Reviews Methods ====================
def reviews_analyze(self, app_id: str, count: int = Config.DEFAULT_REVIEWS_COUNT, lang: str = Config.DEFAULT_LANGUAGE,
country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> List[Dict]:
"""Get user reviews for an app.
Args:
app_id: Google Play app ID
count: Number of reviews to fetch
lang: Language code
country: Country code
sort: Sort order (NEWEST, RELEVANT, RATING)
Returns:
List of review dictionaries
"""
return self.reviews_methods.reviews_analyze(app_id, count, lang, country, sort)
def reviews_get_field(self, app_id: str, field: str, count: int = Config.DEFAULT_REVIEWS_COUNT,
lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> List[Any]:
"""Get single field from reviews.
Args:
app_id: Google Play app ID
field: Field name to retrieve
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
Returns:
List of field values
"""
return self.reviews_methods.reviews_get_field(app_id, field, count, lang, country, sort)
def reviews_get_fields(self, app_id: str, fields: List[str], count: int = Config.DEFAULT_REVIEWS_COUNT,
lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> List[Dict[str, Any]]:
"""Get multiple fields from reviews.
Args:
app_id: Google Play app ID
fields: List of field names
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
Returns:
List of dictionaries with requested fields
"""
return self.reviews_methods.reviews_get_fields(app_id, fields, count, lang, country, sort)
def reviews_print_field(self, app_id: str, field: str, count: int = Config.DEFAULT_REVIEWS_COUNT,
lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> None:
"""Print single field from reviews.
Args:
app_id: Google Play app ID
field: Field name to print
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
"""
return self.reviews_methods.reviews_print_field(app_id, field, count, lang, country, sort)
def reviews_print_fields(self, app_id: str, fields: List[str], count: int = Config.DEFAULT_REVIEWS_COUNT,
lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> None:
"""Print multiple fields from reviews.
Args:
app_id: Google Play app ID
fields: List of field names
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
"""
return self.reviews_methods.reviews_print_fields(app_id, fields, count, lang, country, sort)
def reviews_print_all(self, app_id: str, count: int = Config.DEFAULT_REVIEWS_COUNT, lang: str = Config.DEFAULT_LANGUAGE,
country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> None:
"""Print all reviews as JSON.
Args:
app_id: Google Play app ID
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
"""
return self.reviews_methods.reviews_print_all(app_id, count, lang, country, sort)
# ==================== Developer Methods ====================
def developer_analyze(self, dev_id: str, count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict]:
"""Get all apps from a developer.
Args:
dev_id: Developer ID (numeric or string)
count: Number of apps to return
lang: Language code
country: Country code
Returns:
List of app dictionaries
"""
return self.developer_methods.developer_analyze(dev_id, count, lang, country)
def developer_get_field(self, dev_id: str, field: str, count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Any]:
"""Get single field from developer apps.
Args:
dev_id: Developer ID
field: Field name to retrieve
count: Number of apps
lang: Language code
country: Country code
Returns:
List of field values
"""
return self.developer_methods.developer_get_field(dev_id, field, count, lang, country)
def developer_get_fields(self, dev_id: str, fields: List[str], count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict[str, Any]]:
"""Get multiple fields from developer apps.
Args:
dev_id: Developer ID
fields: List of field names
count: Number of apps
lang: Language code
country: Country code
Returns:
List of dictionaries with requested fields
"""
return self.developer_methods.developer_get_fields(dev_id, fields, count, lang, country)
def developer_print_field(self, dev_id: str, field: str, count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print single field from developer apps.
Args:
dev_id: Developer ID
field: Field name to print
count: Number of apps
lang: Language code
country: Country code
"""
return self.developer_methods.developer_print_field(dev_id, field, count, lang, country)
def developer_print_fields(self, dev_id: str, fields: List[str], count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print multiple fields from developer apps.
Args:
dev_id: Developer ID
fields: List of field names
count: Number of apps
lang: Language code
country: Country code
"""
return self.developer_methods.developer_print_fields(dev_id, fields, count, lang, country)
def developer_print_all(self, dev_id: str, count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all developer apps as JSON.
Args:
dev_id: Developer ID
count: Number of apps
lang: Language code
country: Country code
"""
return self.developer_methods.developer_print_all(dev_id, count, lang, country)
# ==================== Similar Methods ====================
def similar_analyze(self, app_id: str, count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict]:
"""Get similar/competitor apps.
Args:
app_id: Google Play app ID
count: Number of similar apps to return
lang: Language code
country: Country code
Returns:
List of similar app dictionaries
"""
return self.similar_methods.similar_analyze(app_id, count, lang, country)
def similar_get_field(self, app_id: str, field: str, count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Any]:
"""Get single field from similar apps.
Args:
app_id: Google Play app ID
field: Field name to retrieve
count: Number of similar apps
lang: Language code
country: Country code
Returns:
List of field values
"""
return self.similar_methods.similar_get_field(app_id, field, count, lang, country)
def similar_get_fields(self, app_id: str, fields: List[str], count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict[str, Any]]:
"""Get multiple fields from similar apps.
Args:
app_id: Google Play app ID
fields: List of field names
count: Number of similar apps
lang: Language code
country: Country code
Returns:
List of dictionaries with requested fields
"""
return self.similar_methods.similar_get_fields(app_id, fields, count, lang, country)
def similar_print_field(self, app_id: str, field: str, count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print single field from similar apps.
Args:
app_id: Google Play app ID
field: Field name to print
count: Number of similar apps
lang: Language code
country: Country code
"""
return self.similar_methods.similar_print_field(app_id, field, count, lang, country)
def similar_print_fields(self, app_id: str, fields: List[str], count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print multiple fields from similar apps.
Args:
app_id: Google Play app ID
fields: List of field names
count: Number of similar apps
lang: Language code
country: Country code
"""
return self.similar_methods.similar_print_fields(app_id, fields, count, lang, country)
def similar_print_all(self, app_id: str, count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all similar apps as JSON.
Args:
app_id: Google Play app ID
count: Number of similar apps
lang: Language code
country: Country code
"""
return self.similar_methods.similar_print_all(app_id, count, lang, country)
# ==================== List Methods ====================
def list_analyze(self, collection: str = Config.DEFAULT_LIST_COLLECTION, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict]:
"""Get top charts (top free, top paid, top grossing).
Args:
collection: Collection type (TOP_FREE, TOP_PAID, TOP_GROSSING)
category: App category
count: Number of apps to return
lang: Language code
country: Country code
Returns:
List of app dictionaries from top charts
"""
return self.list_methods.list_analyze(collection, category, count, lang, country)
def list_get_field(self, collection: str, field: str, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Any]:
"""Get single field from top charts.
Args:
collection: Collection type
field: Field name to retrieve
category: App category
count: Number of apps
lang: Language code
country: Country code
Returns:
List of field values
"""
return self.list_methods.list_get_field(collection, field, category, count, lang, country)
def list_get_fields(self, collection: str, fields: List[str], category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict[str, Any]]:
"""Get multiple fields from top charts.
Args:
collection: Collection type
fields: List of field names
category: App category
count: Number of apps
lang: Language code
country: Country code
Returns:
List of dictionaries with requested fields
"""
return self.list_methods.list_get_fields(collection, fields, category, count, lang, country)
def list_print_field(self, collection: str, field: str, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print single field from top charts.
Args:
collection: Collection type
field: Field name to print
category: App category
count: Number of apps
lang: Language code
country: Country code
"""
return self.list_methods.list_print_field(collection, field, category, count, lang, country)
def list_print_fields(self, collection: str, fields: List[str], category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print multiple fields from top charts.
Args:
collection: Collection type
fields: List of field names
category: App category
count: Number of apps
lang: Language code
country: Country code
"""
return self.list_methods.list_print_fields(collection, fields, category, count, lang, country)
def list_print_all(self, collection: str = Config.DEFAULT_LIST_COLLECTION, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all top charts as JSON.
Args:
collection: Collection type
category: App category
count: Number of apps
lang: Language code
country: Country code
"""
return self.list_methods.list_print_all(collection, category, count, lang, country)
# ==================== Suggest Methods ====================
def suggest_analyze(self, term: str, count: int = Config.DEFAULT_SUGGEST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[str]:
"""Get search suggestions for a term.
Args:
term: Search term
count: Number of suggestions to return
lang: Language code
country: Country code
Returns:
List of suggestion strings
"""
return self.suggest_methods.suggest_analyze(term, count, lang, country)
def suggest_nested(self, term: str, count: int = Config.DEFAULT_SUGGEST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> Dict[str, List[str]]:
"""Get nested suggestions (suggestions for suggestions).
Args:
term: Search term
count: Number of suggestions
lang: Language code
country: Country code
Returns:
Dictionary mapping terms to their suggestions
"""
return self.suggest_methods.suggest_nested(term, count, lang, country)
def suggest_print_all(self, term: str, count: int = Config.DEFAULT_SUGGEST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all suggestions as JSON.
Args:
term: Search term
count: Number of suggestions
lang: Language code
country: Country code
"""
return self.suggest_methods.suggest_print_all(term, count, lang, country)
def suggest_print_nested(self, term: str, count: int = Config.DEFAULT_SUGGEST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print nested suggestions as JSON.
Args:
term: Search term
count: Number of suggestions
lang: Language code
country: Country code
"""
return self.suggest_methods.suggest_print_nested(term, count, lang, country)
================================================
FILE: gplay_scraper/config.py
================================================
"""Configuration module for GPlay Scraper.
Contains all constants, default values, URLs, and error messages.
"""
import random
from typing import Dict, Any
class Config:
"""Configuration class containing all settings and constants."""
# HTTP request settings
DEFAULT_TIMEOUT = 30 # Request timeout in seconds
RATE_LIMIT_DELAY = 1.0 # Delay between requests in seconds
DEFAULT_RETRY_COUNT = 3 # Number of retries for failed requests
# User agent strings for HTTP requests
USER_AGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.2 Safari/605.1.15",
]
# Google Play Store URLs
PLAY_STORE_BASE_URL = "https://play.google.com"
APP_DETAILS_ENDPOINT = "/store/apps/details" # App details page
BATCHEXECUTE_ENDPOINT = "/_/PlayStoreUi/data/batchexecute" # Batch API endpoint
DEVELOPER_NUMERIC_ENDPOINT = "/store/apps/dev" # Developer page (numeric ID)
DEVELOPER_STRING_ENDPOINT = "/store/apps/developer" # Developer page (string ID)
# Default parameters
DEFAULT_LANGUAGE = "en" # Default language code
DEFAULT_COUNTRY = "" # Default country code
DEFAULT_REVIEWS_SORT = "NEWEST" # Options: NEWEST, RELEVANT, RATING
DEFAULT_HTTP_CLIENT = "requests" # Options: requests, httpx, curl-cffi, tls-client, aiohttp, urllib3, cloudscraper
# Default collection and category for list methods
DEFAULT_LIST_COLLECTION = "TOP_FREE" # Options: TOP_FREE, TOP_PAID, TOP_GROSSING
DEFAULT_LIST_CATEGORY = "APPLICATION" # Default category
# Default count values for different methods
DEFAULT_LIST_COUNT = 100 # Number of apps to fetch from lists
DEFAULT_REVIEWS_COUNT = 100 # Number of reviews to fetch
DEFAULT_REVIEWS_BATCH_SIZE = 50 # Reviews per batch request
DEFAULT_SUGGEST_COUNT = 5 # Number of suggestions to fetch
DEFAULT_SIMILAR_COUNT = 100 # Number of similar apps to fetch
DEFAULT_DEVELOPER_COUNT = 100 # Number of developer apps to fetch
DEFAULT_SEARCH_COUNT = 100 # Number of search results to fetch
# Image size configurations
IMAGE_SIZES = {
"SMALL": "w512", # 512px width
"MEDIUM": "w1024", # 1024px width
"LARGE": "w2048", # 2048px width
"ORIGINAL": "w9999" # Original/max size
}
DEFAULT_IMAGE_SIZE = "MEDIUM" # Default image size
# Error message templates
ERROR_MESSAGES = {
"INVALID_APP_ID": "app_id must be a non-empty string",
"INVALID_DEV_ID": "dev_id must be a non-empty string",
"INVALID_QUERY": "query must be a non-empty string",
"NO_DS5_DATA": "No data found in dataset",
"DS5_NOT_FOUND": "Could not find data",
"JSON_PARSE_FAILED": "Failed to parse JSON: {error}",
"APP_FETCH_FAILED": "Failed to fetch app page for {app_id}: {error}",
"SEARCH_FETCH_FAILED": "Failed to fetch search results for '{query}': {error}",
"REVIEWS_FETCH_FAILED": "Failed to fetch reviews batch for {app_id}: {error}",
"REVIEWS_SCRAPE_FAILED": "Failed to scrape reviews for {app_id}: {error}",
"DEVELOPER_FETCH_FAILED": "Failed to fetch developer page for {dev_id}: {error}",
"CLUSTER_FETCH_FAILED": "Failed to fetch cluster page: {error}",
"LIST_FETCH_FAILED": "Failed to fetch list page: {error}",
"SUGGEST_FETCH_FAILED": "Failed to fetch suggestions for '{term}': {error}",
"RATE_LIMIT_SLEEP": "Rate limiting: sleeping for {sleep_time:.2f} seconds",
"HTTP_CLIENT_NOT_AVAILABLE": "{client} not available",
"HTTP_ERROR": "HTTP {status_code} Error",
"NO_HTTP_CLIENT": "No HTTP client libraries found",
"CLIENT_FAILED_TRYING_NEXT": "{client_type} failed, trying next client: {error}",
"UNKNOWN_CLIENT_TYPE": "Unknown client type: {client_type}",
"APP_NOT_FOUND": "App not found: {app_id}",
"SEARCH_NOT_FOUND": "Search not found: {query}",
"REVIEWS_NOT_FOUND": "Reviews not found for app: {app_id}",
"DEVELOPER_NOT_FOUND": "Developer not found: {dev_id}",
"CLUSTER_NOT_FOUND": "Cluster not found: {cluster_url}",
"LIST_NOT_FOUND": "List not found: {collection}/{category}",
"SUGGEST_NOT_FOUND": "Suggestions not found for: {term}",
"NO_DS3_DATA": "No data found in dataset",
"DS3_NOT_FOUND": "Could not find data",
"DS3_JSON_PARSE_FAILED": "Failed to parse JSON: {error}",
"SEARCH_PAGINATION_FAILED": "Failed to fetch paginated search results: {error}"
}
@classmethod
def get_headers(cls, user_agent: str = None) -> Dict[str, str]:
"""Generate HTTP headers with random or specified user agent.
Args:
user_agent: Optional custom user agent string
Returns:
Dictionary containing HTTP headers
"""
return {
"User-Agent": user_agent or random.choice(cls.USER_AGENTS)
}
@classmethod
def get_image_size(cls, size: str = None) -> str:
"""Get image size parameter.
Args:
size: Size name (SMALL, MEDIUM, LARGE, ORIGINAL) or None for default
Returns:
Image size parameter string
"""
size = size or cls.DEFAULT_IMAGE_SIZE
return cls.IMAGE_SIZES.get(size.upper(), cls.IMAGE_SIZES[cls.DEFAULT_IMAGE_SIZE])
================================================
FILE: gplay_scraper/core/__init__.py
================================================
"""Core module containing all 7 method classes for Google Play Store scraping."""
from .gplay_methods import AppMethods, SearchMethods, ReviewsMethods, DeveloperMethods, SimilarMethods, ListMethods, SuggestMethods
__all__ = ['AppMethods', 'SearchMethods', 'ReviewsMethods', 'DeveloperMethods', 'SimilarMethods', 'ListMethods', 'SuggestMethods']
================================================
FILE: gplay_scraper/core/gplay_methods.py
================================================
"""Method classes for all 7 scraping types.
This module contains 7 method classes, each providing 6 functions (except Suggest with 4):
- analyze(): Get all data
- get_field(): Get single field
- get_fields(): Get multiple fields
- print_field(): Print single field
- print_fields(): Print multiple fields
- print_all(): Print all data as JSON
"""
import json
from typing import Any, List, Dict
import logging
from .gplay_scraper import AppScraper, SearchScraper, ReviewsScraper, DeveloperScraper, SimilarScraper, ListScraper, SuggestScraper
from .gplay_parser import AppParser, SearchParser, ReviewsParser, DeveloperParser, SimilarParser, ListParser, SuggestParser
from ..config import Config
from ..exceptions import InvalidAppIdError, AppNotFoundError
from ..utils.error_handling import comprehensive_error_handler, safe_print
# Configure logging
if not logging.getLogger().handlers:
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
class AppMethods:
"""Methods for extracting app details with 65+ fields."""
def __init__(self, http_client: str = None):
"""Initialize AppMethods with scraper and parser.
Args:
http_client: Optional HTTP client name
"""
self.scraper = AppScraper(http_client=http_client)
self.parser = AppParser()
@comprehensive_error_handler()
def app_analyze(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> Dict:
"""Get complete app data with all 65+ fields.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
Returns:
Dictionary with all app data or None if app not found after retries
Raises:
InvalidAppIdError: If app_id is invalid
"""
if not app_id or not isinstance(app_id, str):
raise InvalidAppIdError(Config.ERROR_MESSAGES["INVALID_APP_ID"])
dataset = self.scraper.scrape_play_store_data(app_id, lang, country)
app_details = self.parser.parse_app_data(dataset, app_id, self.scraper, assets)
return self.parser.format_app_data(app_details)
@comprehensive_error_handler()
def app_get_field(self, app_id: str, field: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> Any:
"""Get single field value from app data.
Args:
app_id: Google Play app ID
field: Field name to retrieve
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
Returns:
Value of the requested field
"""
return self.app_analyze(app_id, lang, country, assets).get(field)
@comprehensive_error_handler()
def app_get_fields(self, app_id: str, fields: List[str], lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> Dict[str, Any]:
"""Get multiple field values from app data.
Args:
app_id: Google Play app ID
fields: List of field names to retrieve
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
Returns:
Dictionary with requested fields and values
"""
data = self.app_analyze(app_id, lang, country, assets)
return {field: data.get(field) for field in fields}
@safe_print()
def app_print_field(self, app_id: str, field: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> None:
"""Print single field value to console.
Args:
app_id: Google Play app ID
field: Field name to print
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
"""
value = self.app_get_field(app_id, field, lang, country, assets)
try:
print(f"{field}: {value}")
except UnicodeEncodeError:
print(f"{field}: {repr(value)}")
@safe_print()
def app_print_fields(self, app_id: str, fields: List[str], lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> None:
"""Print multiple field values to console.
Args:
app_id: Google Play app ID
fields: List of field names to print
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
"""
data = self.app_get_fields(app_id, fields, lang, country, assets)
for field, value in data.items():
try:
print(f"{field}: {value}")
except UnicodeEncodeError:
print(f"{field}: {repr(value)}")
@safe_print()
def app_print_all(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, assets: str = None) -> None:
"""Print all app data as JSON to console.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
assets: Asset size (SMALL, MEDIUM, LARGE, ORIGINAL)
"""
data = self.app_analyze(app_id, lang, country, assets)
try:
print(json.dumps(data, indent=2, ensure_ascii=False))
except UnicodeEncodeError:
print(json.dumps(data, indent=2, ensure_ascii=True))
class SearchMethods:
"""Methods for searching apps by keyword."""
def __init__(self, http_client: str = None):
"""Initialize SearchMethods with scraper and parser.
Args:
http_client: Optional HTTP client name
"""
self.scraper = SearchScraper(http_client=http_client)
self.parser = SearchParser()
@comprehensive_error_handler(return_empty=True)
def search_analyze(self, query: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict]:
"""Search for apps and get complete results with pagination support.
Args:
query: Search query string
count: Number of results to return
lang: Language code
country: Country code
Returns:
List of dictionaries containing app data
Raises:
InvalidAppIdError: If query is invalid
"""
if not query or not isinstance(query, str):
raise InvalidAppIdError(Config.ERROR_MESSAGES["INVALID_QUERY"])
dataset = self.scraper.scrape_play_store_data(query, count, lang, country)
raw_results = self.parser.parse_search_results(dataset, count)
return [self.parser.format_search_result(result) for result in raw_results]
@comprehensive_error_handler(return_empty=True)
def search_get_field(self, query: str, field: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Any]:
"""Get single field from all search results.
Args:
query: Search query string
field: Field name to retrieve
count: Number of results
lang: Language code
country: Country code
Returns:
List of field values from all results
"""
results = self.search_analyze(query, count, lang, country)
return [app.get(field) for app in results]
@comprehensive_error_handler(return_empty=True)
def search_get_fields(self, query: str, fields: List[str], count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict[str, Any]]:
"""Get multiple fields from all search results.
Args:
query: Search query string
fields: List of field names to retrieve
count: Number of results
lang: Language code
country: Country code
Returns:
List of dictionaries with requested fields
"""
results = self.search_analyze(query, count, lang, country)
return [{field: app.get(field) for field in fields} for app in results]
@safe_print()
def search_print_field(self, query: str, field: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print single field from all search results.
Args:
query: Search query string
field: Field name to print
count: Number of results
lang: Language code
country: Country code
"""
values = self.search_get_field(query, field, count, lang, country)
for i, value in enumerate(values):
try:
print(f"{i}. {field}: {value}")
except UnicodeEncodeError:
print(f"{i}. {field}: {repr(value)}")
@safe_print()
def search_print_fields(self, query: str, fields: List[str], count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print multiple fields from all search results.
Args:
query: Search query string
fields: List of field names to print
count: Number of results
lang: Language code
country: Country code
"""
data = self.search_get_fields(query, fields, count, lang, country)
for i, app_data in enumerate(data):
try:
field_str = ', '.join(f'{field}: {value}' for field, value in app_data.items())
print(f"{i}. {field_str}")
except UnicodeEncodeError:
field_str = ', '.join(f'{field}: {repr(value)}' for field, value in app_data.items())
print(f"{i}. {field_str}")
@safe_print()
def search_print_all(self, query: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all search results as JSON.
Args:
query: Search query string
count: Number of results
lang: Language code
country: Country code
"""
results = self.search_analyze(query, count, lang, country)
for i, result in enumerate(results):
try:
print(json.dumps(result, indent=2, ensure_ascii=False))
except UnicodeEncodeError:
print(json.dumps(result, indent=2, ensure_ascii=True))
class ReviewsMethods:
"""Methods for extracting user reviews and ratings."""
def __init__(self, http_client: str = None):
"""Initialize ReviewsMethods with scraper and parser.
Args:
http_client: Optional HTTP client name
"""
self.scraper = ReviewsScraper(http_client=http_client)
self.parser = ReviewsParser()
@comprehensive_error_handler(return_empty=True)
def reviews_analyze(self, app_id: str, count: int = Config.DEFAULT_REVIEWS_COUNT, lang: str = Config.DEFAULT_LANGUAGE,
country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> List[Dict]:
"""Get user reviews for an app.
Args:
app_id: Google Play app ID
count: Number of reviews to fetch
lang: Language code
country: Country code
sort: Sort order (NEWEST, RELEVANT, RATING)
Returns:
List of review dictionaries
Raises:
InvalidAppIdError: If app_id is invalid
"""
if not app_id or not isinstance(app_id, str):
raise InvalidAppIdError(Config.ERROR_MESSAGES["INVALID_APP_ID"])
if count <= 0:
return []
try:
dataset = self.scraper.scrape_reviews_data(app_id, count, lang, country, sort)
reviews_data = self.parser.parse_multiple_responses(dataset)
except Exception as e:
logger.error(Config.ERROR_MESSAGES["REVIEWS_SCRAPE_FAILED"].format(app_id=app_id, error=e))
raise
return self.parser.format_reviews_data(reviews_data)
@comprehensive_error_handler(return_empty=True)
def reviews_get_field(self, app_id: str, field: str, count: int = Config.DEFAULT_REVIEWS_COUNT,
lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> List[Any]:
"""Get single field from all reviews.
Args:
app_id: Google Play app ID
field: Field name to retrieve
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
Returns:
List of field values from all reviews
"""
reviews_data = self.reviews_analyze(app_id, count, lang, country, sort)
return [review.get(field) for review in reviews_data]
@comprehensive_error_handler(return_empty=True)
def reviews_get_fields(self, app_id: str, fields: List[str], count: int = Config.DEFAULT_REVIEWS_COUNT,
lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> List[Dict[str, Any]]:
"""Get multiple fields from all reviews.
Args:
app_id: Google Play app ID
fields: List of field names to retrieve
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
Returns:
List of dictionaries with requested fields
"""
reviews_data = self.reviews_analyze(app_id, count, lang, country, sort)
return [{field: review.get(field) for field in fields} for review in reviews_data]
@safe_print()
def reviews_print_field(self, app_id: str, field: str, count: int = Config.DEFAULT_REVIEWS_COUNT,
lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> None:
"""Print single field from all reviews.
Args:
app_id: Google Play app ID
field: Field name to print
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
"""
field_values = self.reviews_get_field(app_id, field, count, lang, country, sort)
for i, value in enumerate(field_values):
try:
print(f"{i+1}. {field}: {value}")
except UnicodeEncodeError:
print(f"{i+1}. {field}: {repr(value)}")
@safe_print()
def reviews_print_fields(self, app_id: str, fields: List[str], count: int = Config.DEFAULT_REVIEWS_COUNT,
lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> None:
"""Print multiple fields from all reviews.
Args:
app_id: Google Play app ID
fields: List of field names to print
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
"""
reviews_data = self.reviews_get_fields(app_id, fields, count, lang, country, sort)
for i, review in enumerate(reviews_data):
for field, value in review.items():
try:
print(f"{field}: {value}")
except UnicodeEncodeError:
print(f"{field}: {repr(value)}")
@safe_print()
def reviews_print_all(self, app_id: str, count: int = Config.DEFAULT_REVIEWS_COUNT, lang: str = Config.DEFAULT_LANGUAGE,
country: str = Config.DEFAULT_COUNTRY, sort: str = Config.DEFAULT_REVIEWS_SORT) -> None:
"""Print all reviews as JSON.
Args:
app_id: Google Play app ID
count: Number of reviews
lang: Language code
country: Country code
sort: Sort order
"""
reviews_data = self.reviews_analyze(app_id, count, lang, country, sort)
try:
print(json.dumps(reviews_data, indent=2, ensure_ascii=False))
except UnicodeEncodeError:
print(json.dumps(reviews_data, indent=2, ensure_ascii=True))
class DeveloperMethods:
"""Methods for getting all apps from a developer."""
def __init__(self, http_client: str = None):
"""Initialize DeveloperMethods with scraper and parser.
Args:
http_client: Optional HTTP client name
"""
self.scraper = DeveloperScraper(http_client=http_client)
self.parser = DeveloperParser()
@comprehensive_error_handler(return_empty=True)
def developer_analyze(self, dev_id: str, count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict]:
"""Get all apps from a developer.
Args:
dev_id: Developer ID (numeric or string)
count: Number of apps to return
lang: Language code
country: Country code
Returns:
List of app dictionaries
Raises:
InvalidAppIdError: If dev_id is invalid
"""
if not dev_id or not isinstance(dev_id, str):
raise InvalidAppIdError(Config.ERROR_MESSAGES["INVALID_DEV_ID"])
dataset = self.scraper.scrape_play_store_data(dev_id, lang, country)
apps_data = self.parser.parse_developer_data(dataset, dev_id)
return self.parser.format_developer_data(apps_data)[:count]
@comprehensive_error_handler(return_empty=True)
def developer_get_field(self, dev_id: str, field: str, count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Any]:
"""Get single field from all developer apps.
Args:
dev_id: Developer ID
field: Field name to retrieve
count: Number of apps
lang: Language code
country: Country code
Returns:
List of field values from all apps
"""
results = self.developer_analyze(dev_id, count, lang, country)
return [app.get(field) for app in results]
@comprehensive_error_handler(return_empty=True)
def developer_get_fields(self, dev_id: str, fields: List[str], count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict[str, Any]]:
"""Get multiple fields from all developer apps.
Args:
dev_id: Developer ID
fields: List of field names to retrieve
count: Number of apps
lang: Language code
country: Country code
Returns:
List of dictionaries with requested fields
"""
results = self.developer_analyze(dev_id, count, lang, country)
return [{field: app.get(field) for field in fields} for app in results]
@safe_print()
def developer_print_field(self, dev_id: str, field: str, count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print single field from all developer apps.
Args:
dev_id: Developer ID
field: Field name to print
count: Number of apps
lang: Language code
country: Country code
"""
values = self.developer_get_field(dev_id, field, count, lang, country)
for i, value in enumerate(values):
try:
print(f"{i+1}. {field}: {value}")
except UnicodeEncodeError:
print(f"{i+1}. {field}: {repr(value)}")
@safe_print()
def developer_print_fields(self, dev_id: str, fields: List[str], count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print multiple fields from all developer apps.
Args:
dev_id: Developer ID
fields: List of field names to print
count: Number of apps
lang: Language code
country: Country code
"""
data = self.developer_get_fields(dev_id, fields, count, lang, country)
for i, app_data in enumerate(data):
try:
field_str = ', '.join(f'{field}: {value}' for field, value in app_data.items())
print(f"{i+1}. {field_str}")
except UnicodeEncodeError:
field_str = ', '.join(f'{field}: {repr(value)}' for field, value in app_data.items())
print(f"{i+1}. {field_str}")
@safe_print()
def developer_print_all(self, dev_id: str, count: int = Config.DEFAULT_DEVELOPER_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all developer apps as JSON.
Args:
dev_id: Developer ID
count: Number of apps
lang: Language code
country: Country code
"""
results = self.developer_analyze(dev_id, count, lang, country)
try:
print(json.dumps(results, indent=2, ensure_ascii=False))
except UnicodeEncodeError:
print(json.dumps(results, indent=2, ensure_ascii=True))
class SimilarMethods:
"""Methods for finding similar/competitor apps."""
def __init__(self, http_client: str = None):
"""Initialize SimilarMethods with scraper and parser.
Args:
http_client: Optional HTTP client name
"""
self.scraper = SimilarScraper(http_client=http_client)
self.parser = SimilarParser()
@comprehensive_error_handler(return_empty=True)
def similar_analyze(self, app_id: str, count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict]:
"""Get similar/competitor apps.
Args:
app_id: Google Play app ID
count: Number of similar apps to return
lang: Language code
country: Country code
Returns:
List of similar app dictionaries
Raises:
InvalidAppIdError: If app_id is invalid
"""
if not app_id or not isinstance(app_id, str):
raise InvalidAppIdError(Config.ERROR_MESSAGES["INVALID_APP_ID"])
dataset = self.scraper.scrape_play_store_data(app_id, lang, country)
apps_data = self.parser.parse_similar_data(dataset)
return self.parser.format_similar_data(apps_data)[:count]
@comprehensive_error_handler(return_empty=True)
def similar_get_field(self, app_id: str, field: str, count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Any]:
"""Get single field from all similar apps.
Args:
app_id: Google Play app ID
field: Field name to retrieve
count: Number of similar apps
lang: Language code
country: Country code
Returns:
List of field values from all similar apps
"""
results = self.similar_analyze(app_id, count, lang, country)
return [app.get(field) for app in results]
@comprehensive_error_handler(return_empty=True)
def similar_get_fields(self, app_id: str, fields: List[str], count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict[str, Any]]:
"""Get multiple fields from all similar apps.
Args:
app_id: Google Play app ID
fields: List of field names to retrieve
count: Number of similar apps
lang: Language code
country: Country code
Returns:
List of dictionaries with requested fields
"""
results = self.similar_analyze(app_id, count, lang, country)
return [{field: app.get(field) for field in fields} for app in results]
@safe_print()
def similar_print_field(self, app_id: str, field: str, count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print single field from all similar apps.
Args:
app_id: Google Play app ID
field: Field name to print
count: Number of similar apps
lang: Language code
country: Country code
"""
values = self.similar_get_field(app_id, field, count, lang, country)
for i, value in enumerate(values):
try:
print(f"{i+1}. {field}: {value}")
except UnicodeEncodeError:
print(f"{i+1}. {field}: {repr(value)}")
@safe_print()
def similar_print_fields(self, app_id: str, fields: List[str], count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print multiple fields from all similar apps.
Args:
app_id: Google Play app ID
fields: List of field names to print
count: Number of similar apps
lang: Language code
country: Country code
"""
data = self.similar_get_fields(app_id, fields, count, lang, country)
for i, app_data in enumerate(data):
try:
field_str = ', '.join(f'{field}: {value}' for field, value in app_data.items())
print(f"{i+1}. {field_str}")
except UnicodeEncodeError:
field_str = ', '.join(f'{field}: {repr(value)}' for field, value in app_data.items())
print(f"{i+1}. {field_str}")
@safe_print()
def similar_print_all(self, app_id: str, count: int = Config.DEFAULT_SIMILAR_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all similar apps as JSON.
Args:
app_id: Google Play app ID
count: Number of similar apps
lang: Language code
country: Country code
"""
results = self.similar_analyze(app_id, count, lang, country)
try:
print(json.dumps(results, indent=2, ensure_ascii=False))
except UnicodeEncodeError:
print(json.dumps(results, indent=2, ensure_ascii=True))
class ListMethods:
"""Methods for getting top charts (free, paid, grossing)."""
def __init__(self, http_client: str = None):
"""Initialize ListMethods with scraper and parser.
Args:
http_client: Optional HTTP client name
"""
self.scraper = ListScraper(http_client=http_client)
self.parser = ListParser()
@comprehensive_error_handler(return_empty=True)
def list_analyze(self, collection: str = Config.DEFAULT_LIST_COLLECTION, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict]:
"""Get top charts (top free, top paid, top grossing).
Args:
collection: Collection type (TOP_FREE, TOP_PAID, TOP_GROSSING)
category: App category
count: Number of apps to return
lang: Language code
country: Country code
Returns:
List of app dictionaries from top charts
"""
dataset = self.scraper.scrape_play_store_data(collection, category, count, lang, country)
apps_data = self.parser.parse_list_data(dataset, count)
return self.parser.format_list_data(apps_data)
@comprehensive_error_handler(return_empty=True)
def list_get_field(self, collection: str, field: str, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Any]:
"""Get single field from all list apps.
Args:
collection: Collection type
field: Field name to retrieve
category: App category
count: Number of apps
lang: Language code
country: Country code
Returns:
List of field values from all apps
"""
results = self.list_analyze(collection, category, count, lang, country)
return [app.get(field) for app in results]
@comprehensive_error_handler(return_empty=True)
def list_get_fields(self, collection: str, fields: List[str], category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[Dict[str, Any]]:
"""Get multiple fields from all list apps.
Args:
collection: Collection type
fields: List of field names to retrieve
category: App category
count: Number of apps
lang: Language code
country: Country code
Returns:
List of dictionaries with requested fields
"""
results = self.list_analyze(collection, category, count, lang, country)
return [{field: app.get(field) for field in fields} for app in results]
@safe_print()
def list_print_field(self, collection: str, field: str, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print single field from all list apps.
Args:
collection: Collection type
field: Field name to print
category: App category
count: Number of apps
lang: Language code
country: Country code
"""
values = self.list_get_field(collection, field, category, count, lang, country)
for i, value in enumerate(values):
try:
print(f"{i+1}. {field}: {value}")
except UnicodeEncodeError:
print(f"{i+1}. {field}: {repr(value)}")
@safe_print()
def list_print_fields(self, collection: str, fields: List[str], category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print multiple fields from all list apps.
Args:
collection: Collection type
fields: List of field names to print
category: App category
count: Number of apps
lang: Language code
country: Country code
"""
data = self.list_get_fields(collection, fields, category, count, lang, country)
for i, app_data in enumerate(data):
try:
field_str = ', '.join(f'{field}: {value}' for field, value in app_data.items())
print(f"{i+1}. {field_str}")
except UnicodeEncodeError:
field_str = ', '.join(f'{field}: {repr(value)}' for field, value in app_data.items())
print(f"{i+1}. {field_str}")
@safe_print()
def list_print_all(self, collection: str = Config.DEFAULT_LIST_COLLECTION, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all list apps as JSON.
Args:
collection: Collection type
category: App category
count: Number of apps
lang: Language code
country: Country code
"""
results = self.list_analyze(collection, category, count, lang, country)
try:
print(json.dumps(results, indent=2, ensure_ascii=False))
except UnicodeEncodeError:
print(json.dumps(results, indent=2, ensure_ascii=True))
class SuggestMethods:
"""Methods for getting search suggestions and autocomplete."""
def __init__(self, http_client: str = None):
"""Initialize SuggestMethods with scraper and parser.
Args:
http_client: Optional HTTP client name
"""
self.scraper = SuggestScraper(http_client=http_client)
self.parser = SuggestParser()
@comprehensive_error_handler(return_empty=True)
def suggest_analyze(self, term: str, count: int = Config.DEFAULT_SUGGEST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> List[str]:
"""Get search suggestions for a term.
Args:
term: Search term
count: Number of suggestions to return
lang: Language code
country: Country code
Returns:
List of suggestion strings
Raises:
InvalidAppIdError: If term is invalid
"""
if not term or not isinstance(term, str):
raise InvalidAppIdError(Config.ERROR_MESSAGES["INVALID_QUERY"])
dataset = self.scraper.scrape_suggestions(term, lang, country)
suggestions = self.parser.parse_suggestions(dataset)
return self.parser.format_suggestions(suggestions[:count])
@comprehensive_error_handler()
def suggest_nested(self, term: str, count: int = Config.DEFAULT_SUGGEST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> Dict[str, List[str]]:
"""Get nested suggestions (suggestions for suggestions).
Args:
term: Search term
count: Number of suggestions per level
lang: Language code
country: Country code
Returns:
Dictionary mapping suggestions to their nested suggestions
Raises:
InvalidAppIdError: If term is invalid
"""
if not term or not isinstance(term, str):
raise InvalidAppIdError(Config.ERROR_MESSAGES["INVALID_QUERY"])
first_level = self.suggest_analyze(term, count, lang, country)
results = {}
for suggestion in first_level:
second_level = self.suggest_analyze(suggestion, count, lang, country)
results[suggestion] = second_level
return results
@safe_print()
def suggest_print_all(self, term: str, count: int = Config.DEFAULT_SUGGEST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print all suggestions as JSON.
Args:
term: Search term
count: Number of suggestions
lang: Language code
country: Country code
"""
suggestions = self.suggest_analyze(term, count, lang, country)
try:
print(json.dumps(suggestions, indent=2, ensure_ascii=False))
except UnicodeEncodeError:
print(json.dumps(suggestions, indent=2, ensure_ascii=True))
@safe_print()
def suggest_print_nested(self, term: str, count: int = Config.DEFAULT_SUGGEST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> None:
"""Print nested suggestions as JSON.
Args:
term: Search term
count: Number of suggestions per level
lang: Language code
country: Country code
"""
nested = self.suggest_nested(term, count, lang, country)
try:
print(json.dumps(nested, indent=2, ensure_ascii=False))
except UnicodeEncodeError:
print(json.dumps(nested, indent=2, ensure_ascii=True))
================================================
FILE: gplay_scraper/core/gplay_parser.py
================================================
"""Parser classes for extracting and formatting data from raw responses.
This module contains 7 parser classes that handle JSON/HTML parsing and
data formatting for all scraping methods.
"""
import json
import re
from datetime import datetime, timezone
from typing import Dict, Any, List, Optional, Tuple
from ..models.element_specs import ElementSpecs, nested_lookup, format_image_url
from ..utils.helpers import clean_json_string, alternative_json_clean, calculate_app_age, calculate_daily_installs, calculate_monthly_installs, tamp_to_date, get_publisher_country
from ..config import Config
from ..exceptions import DataParsingError
from ..utils.error_handling import handle_parsing_errors
from ..utils.helpers import pho_count, add_count
class AppParser:
"""Parser for extracting and formatting app data."""
@handle_parsing_errors()
def parse_app_data(self, dataset: Dict, app_id: str, scraper=None, assets: str = None) -> Dict[str, Any]:
"""Parse raw app data from dataset with fallback for missing release date.
Args:
dataset: Raw dataset from scraper
app_id: Google Play app ID
scraper: AppScraper instance for fallback requests
Returns:
Dictionary with parsed app details
Raises:
DataParsingError: If parsing fails
"""
ds5_data = dataset.get("ds:5", "")
if not ds5_data:
raise DataParsingError(Config.ERROR_MESSAGES["NO_DS5_DATA"])
json_str_cleaned = clean_json_string(ds5_data)
try:
data = json.loads(json_str_cleaned)
except json.JSONDecodeError as e:
try:
alternative_cleaned = alternative_json_clean(ds5_data)
data = json.loads(alternative_cleaned)
except Exception:
raise DataParsingError(Config.ERROR_MESSAGES["JSON_PARSE_FAILED"].format(error=str(e)))
app_details = {}
for key, spec in ElementSpecs.App.items():
value = spec.extract_content(data.get("data", data))
if key in ["icon", "headerImage", "videoImage"] and value:
app_details[key] = format_image_url(value, assets)
elif key == "screenshots" and value:
app_details[key] = [format_image_url(url, assets) for url in value if url]
else:
app_details[key] = value
app_details['appId'] = app_id
app_details['url'] = f"{Config.PLAY_STORE_BASE_URL}{Config.APP_DETAILS_ENDPOINT}?id={app_id}"
app_details['publisherCountry'] = get_publisher_country(app_details.get('developerPhone'), app_details.get('developerAddress'))
rating_fields = ["released", "score", "ratings", "reviews", "histogram"]
missing_rating_fields = []
for key in rating_fields:
value = app_details.get(key)
if key == "histogram":
if not value or (isinstance(value, list) and all(x == 0 for x in value)):
missing_rating_fields.append(key)
elif not value:
missing_rating_fields.append(key)
if missing_rating_fields and scraper:
try:
country_code = None
phone = app_details.get("developerPhone")
if phone:
country_code = pho_count(phone)
if not country_code:
address = app_details.get("developerAddress")
if address:
country_code = add_count(address)
if country_code:
fallback_dataset = scraper.fetch_fallback_data(app_id, gl=country_code)
suffix = f"fallback_{country_code}"
else:
fallback_dataset = scraper.fetch_fallback_data(app_id, no_locale=True)
suffix = "fallback_no_locale"
if fallback_dataset and fallback_dataset.get("ds:5"):
fallback_cleaned = clean_json_string(fallback_dataset["ds:5"])
try:
fallback_data = json.loads(fallback_cleaned)
for field in missing_rating_fields:
if field in ElementSpecs.App:
spec = ElementSpecs.App[field]
fallback_value = spec.extract_content(fallback_data.get("data", fallback_data))
if fallback_value:
app_details[field] = fallback_value
except:
pass
except:
pass
if not app_details.get("score"):
app_details["score"] = 0
if not app_details.get("ratings"):
app_details["ratings"] = 0
if not app_details.get("reviews"):
app_details["reviews"] = 0
if not app_details.get("installs"):
app_details["installs"] = 0
if not app_details.get("minInstalls"):
app_details["minInstalls"] = 0
current_date = datetime.now(timezone.utc)
release_date_str = app_details.get("released")
if release_date_str:
app_details["appAge"] = calculate_app_age(release_date_str, current_date)
app_details["dailyInstalls"] = calculate_daily_installs(app_details.get("installs"), release_date_str, current_date)
app_details["minDailyInstalls"] = calculate_daily_installs(app_details.get("minInstalls"), release_date_str, current_date)
app_details["realDailyInstalls"] = calculate_daily_installs(app_details.get("realInstalls"), release_date_str, current_date)
app_details["monthlyInstalls"] = calculate_monthly_installs(app_details.get("installs"), release_date_str, current_date)
app_details["minMonthlyInstalls"] = calculate_monthly_installs(app_details.get("minInstalls"), release_date_str, current_date)
app_details["realMonthlyInstalls"] = calculate_monthly_installs(app_details.get("realInstalls"), release_date_str, current_date)
else:
metric_keys = [
"appAge", "dailyInstalls", "minDailyInstalls", "realDailyInstalls",
"monthlyInstalls", "minMonthlyInstalls", "realMonthlyInstalls"
]
for key in metric_keys:
app_details[key] = 0
return app_details
@handle_parsing_errors()
def format_app_data(self, details: dict) -> dict:
"""Format parsed app data into final structure.
Args:
details: Parsed app details
Returns:
Formatted dictionary with all app fields
"""
return {
"appId": details.get("appId"),
"title": details.get("title"),
"summary": details.get("summary"),
"description": details.get("description"),
"genre": details.get("genre"),
"genreId": details.get("genreId"),
"categories": details.get("categories"),
"available": details.get("available"),
"released": details.get("released"),
"appAgeDays": details.get("appAge"),
"lastUpdated": tamp_to_date(details.get("updated")),
"icon": details.get("icon"),
"headerImage": details.get("headerImage"),
"screenshots": details.get("screenshots"),
"video": details.get("video"),
"videoImage": details.get("videoImage"),
"installs": details.get("installs"),
"minInstalls": details.get("minInstalls"),
"realInstalls": details.get("realInstalls"),
"dailyInstalls": details.get("dailyInstalls"),
"minDailyInstalls": details.get("minDailyInstalls"),
"realDailyInstalls": details.get("realDailyInstalls"),
"monthlyInstalls": details.get("monthlyInstalls"),
"minMonthlyInstalls": details.get("minMonthlyInstalls"),
"realMonthlyInstalls": details.get("realMonthlyInstalls"),
"score": details.get("score"),
"ratings": details.get("ratings"),
"reviews": details.get("reviews"),
"histogram": details.get("histogram"),
"adSupported": details.get("adSupported"),
"containsAds": details.get("containsAds"),
"version": details.get("version"),
"androidVersion": details.get("androidVersion"),
"maxAndroidApi": details.get("maxandroidapi"),
"minAndroidApi": details.get("minandroidapi"),
"appBundle": details.get("appBundle"),
"contentRating": details.get("contentRating"),
"contentRatingDescription": details.get("contentRatingDescription"),
"whatsNew": details.get("whatsNew"),
"permissions": details.get("permissions"),
"dataSafety": details.get("dataSafety"),
"price": details.get("price"),
"currency": details.get("currency"),
"free": details.get("free"),
"offersIAP": details.get("offersIAP"),
"inAppProductPrice": details.get("inAppProductPrice"),
"sale": details.get("sale"),
"originalPrice": details.get("originalPrice"),
"developer": details.get("developer"),
"developerId": details.get("developerId"),
"developerEmail": details.get("developerEmail"),
"developerWebsite": details.get("developerWebsite"),
"developerAddress": details.get("developerAddress"),
"developerPhone": details.get("developerPhone"),
"publisherCountry": details.get("publisherCountry"),
"privacyPolicy": details.get("privacyPolicy"),
"appUrl": details.get("url"),
}
class SearchParser:
"""Parser for extracting and formatting search results."""
@handle_parsing_errors(return_empty=True)
def parse_search_results(self, dataset: Dict, count: int) -> List[Dict]:
"""Parse search results from dataset.
Args:
dataset: Raw dataset from scraper
count: Maximum number of results to parse
Returns:
List of parsed search result dictionaries
"""
if "ds:1" not in dataset:
return []
search_data = nested_lookup(dataset.get("ds:1", {}), [0, 1, 0, 0, 0])
if not search_data:
return []
results = []
n_apps = min(len(search_data), count)
for i in range(n_apps):
app = self.extract_search_result(search_data[i])
if app:
results.append(app)
return results[:count]
@handle_parsing_errors()
def extract_search_result(self, data) -> Dict:
"""Extract single search result from raw data.
Args:
data: Raw search result data
Returns:
Dictionary with extracted search result or None if extraction fails
"""
try:
result = {}
for key, spec in ElementSpecs.Search.items():
result[key] = spec.extract_content(data)
return result
except Exception:
return None
@handle_parsing_errors()
def format_search_result(self, result: dict) -> dict:
"""Format parsed search result into final structure.
Args:
result: Parsed search result
Returns:
Formatted dictionary with search result fields
"""
return {
"appId": result.get("appId"),
"title": result.get("title"),
"description": result.get("summary"),
"icon": result.get("icon"),
"developer": result.get("developer"),
"score": result.get("score"),
"scoreText": result.get("scoreText"),
"currency": result.get("currency"),
"price": result.get("price"),
"free": result.get("free"),
"url": result.get("url"),
}
@handle_parsing_errors()
def extract_pagination_token(self, dataset: Dict) -> str:
"""Extract pagination token from search dataset.
Args:
dataset: Search dataset
Returns:
Pagination token or None
"""
sections = nested_lookup(dataset.get("ds:1", {}), [0, 1, 0, 0])
if not sections:
return None
for section in sections:
if isinstance(section, list) and len(section) > 1:
potential_token = nested_lookup(section, [1])
if isinstance(potential_token, str):
return potential_token
return None
@handle_parsing_errors()
def parse_html_content(self, html_content: str) -> Dict:
"""Extract datasets from search page HTML.
Args:
html_content: HTML content of search page
Returns:
Dictionary containing all datasets
Raises:
DataParsingError: If no datasets found
"""
script_regex = re.compile(r"AF_initDataCallback[\s\S]*? Tuple[List[Dict], Optional[str]]:
"""Parse reviews from API response content.
Args:
content: Raw API response content
Returns:
Tuple of (list of review dictionaries, next page token)
"""
if not content or not isinstance(content, str):
return [], None
regex = re.compile(r"\)]}'\n\n([\s\S]+)")
matches = regex.findall(content)
if not matches:
return [], None
try:
data = json.loads(matches[0])
if not data or len(data) == 0 or len(data[0]) < 3:
return [], None
reviews_data = json.loads(data[0][2])
# Handle case where reviews_data is None or empty
if not reviews_data:
return [], None
next_token = None
try:
if (isinstance(reviews_data, list) and len(reviews_data) >= 2 and
reviews_data[-2] and isinstance(reviews_data[-2], list) and len(reviews_data[-2]) > 0):
potential_token = reviews_data[-2][-1]
if isinstance(potential_token, str):
next_token = potential_token
except (IndexError, TypeError, AttributeError):
pass
# Check if we have actual reviews data
if (not isinstance(reviews_data, list) or len(reviews_data) == 0 or
not isinstance(reviews_data[0], list) or len(reviews_data[0]) == 0):
return [], None
reviews = []
for review_raw in reviews_data[0]:
if review_raw: # Make sure review_raw is not None
review = self.extract_review_data(review_raw)
if review:
reviews.append(review)
return reviews, next_token
except (json.JSONDecodeError, IndexError, KeyError, TypeError, AttributeError):
return [], None
@handle_parsing_errors()
def extract_review_data(self, review_raw) -> Optional[Dict]:
"""Extract single review from raw data.
Args:
review_raw: Raw review data array
Returns:
Dictionary with extracted review data or None if extraction fails
"""
try:
review = {
"reviewId": review_raw[0] if len(review_raw) > 0 else None,
"userName": review_raw[1][0] if len(review_raw) > 1 and review_raw[1] else None,
"userImage": None,
"content": review_raw[4] if len(review_raw) > 4 else None,
"score": review_raw[2] if len(review_raw) > 2 else None,
"thumbsUpCount": review_raw[6] if len(review_raw) > 6 else None,
"at": datetime.fromtimestamp(review_raw[5][0]).isoformat() if len(review_raw) > 5 and review_raw[5] else None,
"appVersion": review_raw[10] if len(review_raw) > 10 else None,
}
try:
if len(review_raw) > 1 and review_raw[1] and len(review_raw[1]) > 1 and review_raw[1][1]:
review["userImage"] = review_raw[1][1][3][2]
except:
pass
return review
except Exception:
return None
@handle_parsing_errors(return_empty=True)
def parse_multiple_responses(self, dataset: Dict) -> List[Dict]:
"""Parse multiple review responses.
Args:
dataset: Dataset containing multiple review responses
Returns:
List of all parsed reviews
"""
if not dataset or not isinstance(dataset, dict):
return []
responses = dataset.get("reviews", [])
if not responses or not isinstance(responses, list):
return []
all_reviews = []
for response in responses:
if response and isinstance(response, str):
try:
reviews, _ = self.parse_reviews_response(response)
if reviews: # Only extend if we got actual reviews
all_reviews.extend(reviews)
except Exception:
continue # Skip this response if it fails
return all_reviews
@handle_parsing_errors(return_empty=True)
def format_reviews_data(self, reviews_data: List[Dict]) -> List[Dict]:
"""Format parsed reviews into final structure.
Args:
reviews_data: List of parsed reviews
Returns:
List of formatted review dictionaries
"""
formatted_reviews = []
for review in reviews_data:
formatted_review = {
"reviewId": review.get("reviewId"),
"userName": review.get("userName"),
"userImage": review.get("userImage"),
"score": review.get("score"),
"content": review.get("content"),
"thumbsUpCount": review.get("thumbsUpCount"),
"appVersion": review.get("appVersion"),
"at": review.get("at"),
}
formatted_reviews.append(formatted_review)
return formatted_reviews
class DeveloperParser:
"""Parser for extracting and formatting developer apps."""
@handle_parsing_errors(return_empty=True)
def parse_developer_data(self, dataset: Dict, dev_id: str) -> List[Dict]:
"""Parse developer apps from dataset.
Args:
dataset: Raw dataset from scraper
dev_id: Developer ID (numeric or string)
Returns:
List of parsed app dictionaries
Raises:
DataParsingError: If parsing fails
"""
ds3_data = dataset.get("ds:3", "")
if not ds3_data:
raise DataParsingError(Config.ERROR_MESSAGES["NO_DS3_DATA"])
json_str_cleaned = clean_json_string(ds3_data)
try:
data = json.loads(json_str_cleaned)
except json.JSONDecodeError as e:
try:
alternative_cleaned = alternative_json_clean(ds3_data)
data = json.loads(alternative_cleaned)
except Exception:
raise DataParsingError(Config.ERROR_MESSAGES["DS3_JSON_PARSE_FAILED"].format(error=str(e)))
# Navigate to apps array based on dev_id type
is_numeric = dev_id.isdigit()
if is_numeric:
apps_path = [0, 1, 0, 21, 0]
else:
apps_path = [0, 1, 0, 22, 0]
apps_data = nested_lookup(data.get("data", data), apps_path)
if not apps_data:
return []
apps = []
for app_data in apps_data:
app_details = {}
for key, spec in ElementSpecs.Developer.items():
app_details[key] = spec.extract_content(app_data)
if app_details.get("title"):
apps.append(app_details)
return apps
@handle_parsing_errors(return_empty=True)
def format_developer_data(self, apps_data: List[Dict]) -> List[Dict]:
"""Format parsed developer apps into final structure.
Args:
apps_data: List of parsed apps
Returns:
List of formatted app dictionaries
"""
formatted_apps = []
for app in apps_data:
formatted_app = {
"appId": app.get("appId"),
"title": app.get("title"),
"description": app.get("description"),
"icon": app.get("icon"),
"developer": app.get("developer"),
"score": app.get("score"),
"scoreText": app.get("scoreText"),
"currency": app.get("currency"),
"price": app.get("price"),
"free": app.get("free"),
"url": app.get("url"),
}
formatted_apps.append(formatted_app)
return formatted_apps
class SimilarParser:
"""Parser for extracting and formatting similar apps."""
@handle_parsing_errors(return_empty=True)
def parse_similar_data(self, dataset: Dict) -> List[Dict]:
"""Parse similar apps from dataset.
Args:
dataset: Raw dataset from scraper
Returns:
List of parsed similar app dictionaries
"""
ds3_data = dataset.get("ds:3", "")
if not ds3_data:
return []
json_str_cleaned = clean_json_string(ds3_data)
try:
data = json.loads(json_str_cleaned)
except json.JSONDecodeError as e:
try:
alternative_cleaned = alternative_json_clean(ds3_data)
data = json.loads(alternative_cleaned)
except Exception:
return []
apps_data = nested_lookup(data.get("data", data), [0, 1, 0, 21, 0])
if not apps_data:
return []
apps = []
for app_data in apps_data:
app_details = {}
for key, spec in ElementSpecs.Similar.items():
app_details[key] = spec.extract_content(app_data)
if app_details.get("title"):
apps.append(app_details)
return apps
@handle_parsing_errors(return_empty=True)
def format_similar_data(self, apps_data: List[Dict]) -> List[Dict]:
"""Format parsed similar apps into final structure.
Args:
apps_data: List of parsed apps
Returns:
List of formatted app dictionaries
"""
formatted_apps = []
for app in apps_data:
formatted_app = {
"appId": app.get("appId"),
"title": app.get("title"),
"description": app.get("description"),
"icon": app.get("icon"),
"developer": app.get("developer"),
"score": app.get("score"),
"scoreText": app.get("scoreText"),
"currency": app.get("currency"),
"price": app.get("price"),
"free": app.get("free"),
"url": app.get("url"),
}
formatted_apps.append(formatted_app)
return formatted_apps
class ListParser:
"""Parser for extracting and formatting top chart apps."""
@handle_parsing_errors(return_empty=True)
def parse_list_data(self, dataset: Dict, count: int) -> List[Dict]:
"""Parse top chart apps from dataset.
Args:
dataset: Raw dataset from scraper
count: Maximum number of apps to parse
Returns:
List of parsed app dictionaries
"""
collection_data = dataset.get("collection_data")
if not collection_data:
return []
apps_data = nested_lookup(collection_data, [0, 1, 0, 28, 0])
if not apps_data:
return []
apps = []
for app_data in apps_data[:count]:
app_details = {}
for key, spec in ElementSpecs.List.items():
app_details[key] = spec.extract_content(app_data)
if app_details.get("title"):
apps.append(app_details)
return apps
@handle_parsing_errors(return_empty=True)
def format_list_data(self, apps_data: List[Dict]) -> List[Dict]:
"""Format parsed list apps into final structure.
Args:
apps_data: List of parsed apps
Returns:
List of formatted app dictionaries
"""
formatted_apps = []
for app in apps_data:
formatted_app = {
"appId": app.get("appId"),
"title": app.get("title"),
"description": app.get("description"),
"icon": app.get("icon"),
"screenshots": app.get("screenshots"),
"developer": app.get("developer"),
"genre": app.get("genre"),
"score": app.get("score"),
"scoreText": app.get("scoreText"),
"installs": app.get("installs"),
"currency": app.get("currency"),
"price": app.get("price"),
"free": app.get("free"),
"url": app.get("url"),
}
formatted_apps.append(formatted_app)
return formatted_apps
class SuggestParser:
"""Parser for extracting and formatting search suggestions."""
@handle_parsing_errors(return_empty=True)
def parse_suggestions(self, dataset: Dict) -> List[str]:
"""Parse suggestions from dataset.
Args:
dataset: Raw dataset from scraper
Returns:
List of suggestion strings
"""
return dataset.get("suggestions", [])
@handle_parsing_errors(return_empty=True)
def format_suggestions(self, suggestions: List[str]) -> List[str]:
"""Format suggestions (pass-through for strings).
Args:
suggestions: List of suggestion strings
Returns:
Same list of suggestion strings
"""
return suggestions
================================================
FILE: gplay_scraper/core/gplay_scraper.py
================================================
import json
import re
import logging
from typing import Dict
from ..utils.http_client import HttpClient
from ..config import Config
from ..exceptions import DataParsingError, InvalidAppIdError, AppNotFoundError
from ..utils.error_handling import handle_network_errors, handle_parsing_errors, validate_inputs
from urllib.parse import quote
from .gplay_parser import SearchParser
from ..utils.constants import SORT_NAMES, CLUSTER_NAMES
logger = logging.getLogger(__name__)
class AppScraper:
"""Scraper for fetching app details from Google Play Store.
Handles the extraction of comprehensive app information including ratings,
reviews, install counts, pricing, and metadata. Supports fallback data
fetching when primary requests fail to retrieve certain fields.
Features:
- Primary app data extraction from HTML pages
- Fallback data fetching for missing fields (release dates, ratings)
- Multiple locale support for regional data
- Automatic retry with different parameters
"""
def __init__(self, rate_limit_delay: float = None, http_client: str = None):
"""Initialize AppScraper with HTTP client.
Args:
rate_limit_delay: Delay between requests in seconds (default: 1.0)
http_client: HTTP client to use (requests, curl_cffi, etc.)
"""
self.http_client = HttpClient(rate_limit_delay, http_client)
def fetch_playstore_page(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch app page HTML from Google Play Store.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
Returns:
HTML content of app page
"""
return self.http_client.fetch_app_page(app_id, lang, country)
def fetch_fallback_data(self, app_id: str, gl: str = None, no_locale: bool = False) -> Dict:
"""Fetch app data with specific country or without locale parameters.
Args:
app_id: Google Play app ID
gl: Country code for fallback request
no_locale: If True, fetch without hl and gl parameters
Returns:
Dictionary containing ds:5 dataset from fallback request
"""
if no_locale:
html_content = self.http_client.fetch_app_page_no_locale(app_id)
elif gl:
html_content = self.http_client.fetch_app_page(app_id, lang=Config.DEFAULT_LANGUAGE, country=gl)
else:
html_content = self.http_client.fetch_app_page_no_locale(app_id)
ds_match = re.search(r'AF_initDataCallback\s*\(\s*({\s*key:\s*["\']ds:5["\'][\s\S]*?})\s*\)\s*;', html_content, re.DOTALL)
if ds_match:
ds5_data = ds_match.group(1)
else:
all_callbacks = re.findall(r'AF_initDataCallback\s*\(\s*({[\s\S]*?})\s*\)\s*;', html_content, re.DOTALL)
ds5_data = ""
for callback in all_callbacks:
if "'ds:5'" in callback or '"ds:5"' in callback:
ds5_data = callback
break
return {"ds:5": ds5_data} if ds5_data else None
@validate_inputs()
@handle_network_errors()
@handle_parsing_errors()
def scrape_play_store_data(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> Dict:
"""Extract dataset from app page HTML.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
Returns:
Dictionary containing ds:5 dataset
Raises:
DataParsingError: If dataset not found
AppNotFoundError: If app not found
"""
html_content = self.fetch_playstore_page(app_id, lang, country)
ds_match = re.search(r'AF_initDataCallback\s*\(\s*({\s*key:\s*["\']ds:5["\'][\s\S]*?})\s*\)\s*;', html_content, re.DOTALL)
if ds_match:
ds5_data = ds_match.group(1)
else:
all_callbacks = re.findall(r'AF_initDataCallback\s*\(\s*({[\s\S]*?})\s*\)\s*;', html_content, re.DOTALL)
ds5_data = ""
for callback in all_callbacks:
if "'ds:5'" in callback or '"ds:5"' in callback:
ds5_data = callback
break
if not ds5_data:
raise DataParsingError(Config.ERROR_MESSAGES["DS5_NOT_FOUND"])
return {"ds:5": ds5_data, "fallback_needed": False}
class SearchScraper:
"""Scraper for fetching search results from Google Play Store.
Handles app search functionality with support for pagination to retrieve
large numbers of search results. Integrates with SearchParser for data extraction.
Features:
- Initial search page fetching
- Automatic pagination for large result sets
- Token-based continuation for additional results
- Configurable result limits
"""
def __init__(self, rate_limit_delay: float = None, http_client: str = None):
"""Initialize SearchScraper with HTTP client and parser.
Args:
rate_limit_delay: Delay between requests in seconds
http_client: HTTP client to use for requests
"""
self.http_client = HttpClient(rate_limit_delay, http_client)
self.parser = SearchParser()
def fetch_playstore_search(self, query: str, count: int, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch search page HTML from Google Play Store.
Args:
query: Search query string
count: Number of results needed
lang: Language code
country: Country code
Returns:
HTML content of search page
Raises:
InvalidAppIdError: If query is invalid
"""
if not query or not isinstance(query, str):
raise InvalidAppIdError(Config.ERROR_MESSAGES["INVALID_QUERY"])
if count <= 0:
return ""
return self.http_client.fetch_search_page(query=query, lang=lang, country=country)
@validate_inputs()
@handle_network_errors()
@handle_parsing_errors()
def scrape_play_store_data(self, query: str, count: int = Config.DEFAULT_SEARCH_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> Dict:
"""Scrape search results with automatic pagination support.
Args:
query: Search query string
count: Total number of results to fetch
lang: Language code
country: Country code
Returns:
Dictionary containing all search results
Raises:
DataParsingError: If parsing fails
"""
html_content = self.fetch_playstore_search(query, count, lang, country)
dataset = self.parser.parse_html_content(html_content)
if count <= Config.DEFAULT_SEARCH_COUNT // 5:
return dataset
token = self.parser.extract_pagination_token(dataset)
all_results = []
initial_results = self._get_nested_value(dataset.get("ds:1", []), [0, 1, 0, 0, 0], [])
all_results.extend(initial_results)
while len(all_results) < count and token:
needed = min(Config.DEFAULT_REVIEWS_BATCH_SIZE * 2, count - len(all_results))
try:
response_text = self.http_client.fetch_search_page(token=token, needed=needed, lang=lang, country=country)
data = json.loads(response_text[5:])
parsed_data = json.loads(data[0][2])
if parsed_data:
paginated_results = self._get_nested_value(parsed_data, [0, 0, 0], [])
all_results.extend(paginated_results)
token = self._get_nested_value(parsed_data, [0, 0, 7, 1])
else:
break
except (json.JSONDecodeError, IndexError, KeyError, Exception):
break
if "ds:1" in dataset:
dataset["ds:1"][0][1][0][0][0] = all_results[:count]
return dataset
def _get_nested_value(self, data, path, default=None):
"""Safely get nested value from data structure.
Args:
data: Data structure to traverse
path: List of keys/indices to follow
default: Default value if path not found
Returns:
Value at path or default
"""
try:
for key in path:
data = data[key]
return data
except (KeyError, IndexError, TypeError):
return default
class ReviewsScraper:
"""Scraper for fetching user reviews from Google Play Store.
Handles extraction of user reviews using Google Play's internal API.
Supports different sorting options and pagination for large review sets.
Features:
- Multiple sort orders (newest, relevant, rating)
- Batch processing for large review counts
- Pagination token management
- Configurable batch sizes
"""
def __init__(self, rate_limit_delay: float = None, http_client: str = None):
"""Initialize ReviewsScraper with HTTP client.
Args:
rate_limit_delay: Delay between requests in seconds
http_client: HTTP client to use for API requests
"""
self.http_client = HttpClient(rate_limit_delay, http_client)
def fetch_reviews_batch(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY,
sort: int = Config.DEFAULT_REVIEWS_SORT, batch_count: int = Config.DEFAULT_REVIEWS_BATCH_SIZE, token: str = None) -> str:
"""Fetch single batch of reviews from API.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
sort: Sort order (NEWEST, RELEVANT, RATING)
batch_count: Number of reviews per batch
token: Pagination token for next batch
Returns:
Raw API response content
"""
sort_value = SORT_NAMES.get(sort, sort) if isinstance(sort, str) else sort
return self.http_client.fetch_reviews_batch(app_id, lang, country, sort_value, batch_count, token)
@validate_inputs()
@handle_network_errors()
@handle_parsing_errors()
def scrape_reviews_data(self, app_id: str, count: int = Config.DEFAULT_REVIEWS_COUNT, lang: str = Config.DEFAULT_LANGUAGE,
country: str = Config.DEFAULT_COUNTRY, sort: int = Config.DEFAULT_REVIEWS_SORT) -> Dict:
"""Scrape multiple batches of reviews.
Args:
app_id: Google Play app ID
count: Total number of reviews to fetch
lang: Language code
country: Country code
sort: Sort order
Returns:
Dictionary containing all review responses
"""
all_responses = []
token = None
batch_size = Config.DEFAULT_REVIEWS_BATCH_SIZE
while len(all_responses) * batch_size < count:
remaining = count - (len(all_responses) * batch_size)
fetch_count = min(batch_size, remaining)
response = self.fetch_reviews_batch(app_id, lang, country, sort, fetch_count, token)
if not response:
break
all_responses.append(response)
try:
regex = re.compile(r"\)]}'\n\n([\s\S]+)")
matches = regex.findall(response)
if matches:
data = json.loads(matches[0])
parsed_data = json.loads(data[0][2])
# Check if we got any reviews in this batch
if not parsed_data or len(parsed_data) == 0 or (len(parsed_data) > 0 and len(parsed_data[0]) == 0):
break
# Extract next token safely
try:
if len(parsed_data) >= 2 and parsed_data[-2] and len(parsed_data[-2]) > 0:
token = parsed_data[-2][-1]
else:
token = None
except (IndexError, TypeError, AttributeError):
token = None
if not token or isinstance(token, list) or not isinstance(token, str):
break
else:
break
except (json.JSONDecodeError, IndexError, KeyError, TypeError):
break
return {"reviews": all_responses if all_responses else []}
class DeveloperScraper:
"""Scraper for fetching developer portfolio from Google Play Store.
Extracts all apps published by a specific developer, supporting both
numeric developer IDs and string-based developer names.
Features:
- Numeric developer ID support (e.g., '5700313618786177705')
- String developer name support (e.g., 'Google LLC')
- Complete app portfolio extraction
- Developer metadata collection
"""
def __init__(self, rate_limit_delay: float = None, http_client: str = None):
"""Initialize DeveloperScraper with HTTP client.
Args:
rate_limit_delay: Delay between requests in seconds
http_client: HTTP client to use for requests
"""
self.http_client = HttpClient(rate_limit_delay, http_client)
def fetch_developer_page(self, dev_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch developer page HTML from Google Play Store.
Args:
dev_id: Developer ID (numeric or string)
lang: Language code
country: Country code
Returns:
HTML content of developer page
"""
return self.http_client.fetch_developer_page(dev_id, lang, country)
@validate_inputs()
@handle_network_errors()
@handle_parsing_errors()
def scrape_play_store_data(self, dev_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> Dict:
"""Extract dataset from developer page HTML.
Args:
dev_id: Developer ID
lang: Language code
country: Country code
Returns:
Dictionary containing ds:3 dataset and dev_id
Raises:
DataParsingError: If dataset not found
"""
html_content = self.fetch_developer_page(dev_id, lang, country)
ds_match = re.search(r'AF_initDataCallback\s*\(\s*({\s*key:\s*["\']ds:3["\'][\s\S]*?})\s*\)\s*;', html_content, re.DOTALL)
if ds_match:
ds3_data = ds_match.group(1)
else:
all_callbacks = re.findall(r'AF_initDataCallback\s*\(\s*({[\s\S]*?})\s*\)\s*;', html_content, re.DOTALL)
ds3_data = ""
for callback in all_callbacks:
if "'ds:3'" in callback or '"ds:3"' in callback:
ds3_data = callback
break
if not ds3_data:
raise DataParsingError(Config.ERROR_MESSAGES["DS3_NOT_FOUND"])
return {"ds:3": ds3_data, "dev_id": dev_id}
class SimilarScraper:
"""Scraper for fetching similar apps from Google Play Store.
Extracts similar/related apps by finding cluster URLs from app pages
and fetching the corresponding collection pages.
Features:
- Cluster URL extraction from app pages
- Similar app collection fetching
- Related app recommendations
- Competitive analysis data
"""
def __init__(self, rate_limit_delay: float = None, http_client: str = None):
"""Initialize SimilarScraper with HTTP client.
Args:
rate_limit_delay: Delay between requests in seconds
http_client: HTTP client to use for requests
"""
self.http_client = HttpClient(rate_limit_delay, http_client)
def fetch_similar_page(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch app page HTML to extract similar apps cluster URL.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
Returns:
HTML content of app page
"""
return self.http_client.fetch_app_page(app_id, lang, country)
@validate_inputs()
@handle_network_errors()
@handle_parsing_errors()
def scrape_play_store_data(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> Dict:
"""Extract similar apps dataset from cluster page.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
Returns:
Dictionary containing ds:3 dataset
Raises:
DataParsingError: If dataset not found
"""
html_content = self.fetch_similar_page(app_id, lang, country)
pattern1 = r'"(/store/apps/collection/cluster\?gsr=[^&]+)"'
matches1 = re.findall(pattern1, html_content)
pattern2 = r'"(/store/apps/collection/cluster\?gsr=[^"]+)"'
matches2 = re.findall(pattern2, html_content)
all_matches = list(set(matches1 + matches2))
if not all_matches:
return {"ds:3": None}
cluster_url = all_matches[0].replace('&', '&')
cluster_html = self.http_client.fetch_cluster_page(cluster_url, lang, country)
ds_match = re.search(r'AF_initDataCallback\s*\(\s*({\s*key:\s*["\']ds:3["\'][\s\S]*?})\s*\)\s*;', cluster_html, re.DOTALL)
if ds_match:
ds3_data = ds_match.group(1)
else:
all_callbacks = re.findall(r'AF_initDataCallback\s*\(\s*({[\s\S]*?})\s*\)\s*;', cluster_html, re.DOTALL)
ds3_data = ""
for callback in all_callbacks:
if "'ds:3'" in callback or '"ds:3"' in callback:
ds3_data = callback
break
if not ds3_data:
raise DataParsingError(Config.ERROR_MESSAGES["DS3_NOT_FOUND"])
return {"ds:3": ds3_data}
class ListScraper:
"""Scraper for fetching top charts from Google Play Store.
Handles extraction of ranked app lists including top free, top paid,
and top grossing apps across different categories.
Features:
- Multiple collection types (free, paid, grossing)
- Category-specific charts (games, social, productivity, etc.)
- Configurable result counts
- Regional chart variations
"""
def __init__(self, rate_limit_delay: float = None, http_client: str = None):
"""Initialize ListScraper with HTTP client.
Args:
rate_limit_delay: Delay between requests in seconds
http_client: HTTP client to use for API requests
"""
self.http_client = HttpClient(rate_limit_delay, http_client)
@handle_network_errors()
@handle_parsing_errors()
def scrape_play_store_data(self, collection: str, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> Dict:
"""Scrape top charts data from Google Play Store.
Args:
collection: Collection type (TOP_FREE, TOP_PAID, TOP_GROSSING)
category: App category (e.g., GAME, SOCIAL)
count: Number of apps to fetch
lang: Language code
country: Country code
Returns:
Dictionary containing collection data
Raises:
DataParsingError: If JSON parsing fails
"""
cluster = CLUSTER_NAMES.get(collection, collection)
response_text = self.http_client.fetch_list_page(cluster, category, count, lang, country)
try:
lines = response_text.strip().split('\n')
data = json.loads(lines[2])
collection_data = json.loads(data[0][2])
return {"collection_data": collection_data}
except (json.JSONDecodeError, IndexError, KeyError) as e:
raise DataParsingError(Config.ERROR_MESSAGES["JSON_PARSE_FAILED"].format(error=str(e)))
class SuggestScraper:
"""Scraper for fetching search suggestions from Google Play Store.
Provides autocomplete functionality for search terms, useful for
keyword research and ASO (App Store Optimization) analysis.
Features:
- Real-time search suggestions
- Keyword research capabilities
- ASO optimization data
- Popular search term discovery
"""
def __init__(self, rate_limit_delay: float = None, http_client: str = None):
"""Initialize SuggestScraper with HTTP client.
Args:
rate_limit_delay: Delay between requests in seconds
http_client: HTTP client to use for API requests
"""
self.http_client = HttpClient(rate_limit_delay, http_client)
@validate_inputs()
@handle_network_errors()
@handle_parsing_errors()
def scrape_suggestions(self, term: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> Dict:
"""Scrape search suggestions from Google Play Store.
Args:
term: Search term for suggestions
lang: Language code
country: Country code
Returns:
Dictionary containing list of suggestions
Raises:
DataParsingError: If JSON parsing fails
"""
if not term:
return {"suggestions": []}
response_text = self.http_client.fetch_suggest_page(term, lang, country)
try:
input_data = json.loads(response_text[5:])
data = json.loads(input_data[0][2])
if data is None:
return {"suggestions": []}
suggestions = [s[0] for s in data[0][0]]
return {"suggestions": suggestions}
except (json.JSONDecodeError, IndexError, KeyError, TypeError) as e:
raise DataParsingError(Config.ERROR_MESSAGES["JSON_PARSE_FAILED"].format(error=str(e)))
================================================
FILE: gplay_scraper/exceptions.py
================================================
"""Custom exceptions for GPlay Scraper.
This module defines all custom exceptions used throughout the library.
"""
class GPlayScraperError(Exception):
"""Base exception for all GPlay Scraper errors."""
pass
class InvalidAppIdError(GPlayScraperError):
"""Raised when an invalid app ID, dev ID, or query is provided."""
pass
class AppNotFoundError(GPlayScraperError):
"""Raised when an app, developer, or resource is not found (404 error)."""
pass
class RateLimitError(GPlayScraperError):
"""Raised when rate limiting is triggered by Google Play Store."""
pass
class NetworkError(GPlayScraperError):
"""Raised when network requests fail."""
pass
class DataParsingError(GPlayScraperError):
"""Raised when parsing JSON or HTML data fails."""
pass
================================================
FILE: gplay_scraper/models/__init__.py
================================================
================================================
FILE: gplay_scraper/models/element_specs.py
================================================
"""Element specifications for data extraction from Google Play Store.
This module defines ElementSpec class and ElementSpecs for all 7 method types.
Each spec defines how to extract specific fields from raw JSON data.
"""
from typing import Any, Callable, List, Optional, Dict, Union
import html
from datetime import datetime
from ..utils.helpers import unescape_text
from ..config import Config
def parse_permissions(perms_data: Any) -> Dict[str, List[str]]:
"""Parse permissions from various Google Play Store data formats.
Google Play Store uses complex nested data structures for app permissions.
This function handles all known formats and extracts human-readable permission
descriptions organized by category.
Data Structure Patterns:
- Format 1: [[[category, [...], [[null, description]], [...]]]]
- Format 2: [[category, [...], [description1, description2]]]
- Format 3: Mixed with "Other" category for uncategorized permissions
- Format 4: Empty/null data for apps with no permissions
Args:
perms_data: Raw permissions data from Play Store JSON (nested lists/dicts)
Returns:
Dictionary mapping permission categories to lists of permission descriptions
Example: {"Location": ["GPS access"], "Storage": ["Read files", "Write files"]}
Examples:
>>> parse_permissions(None)
{}
>>> parse_permissions([[[["Location", [...], [[null, "GPS access"]], [...]]]])
{"Location": ["GPS access"]}
>>> parse_permissions([["Storage", [...], ["Read files", "Write files"]]])
{"Storage": ["Read files", "Write files"]}
"""
if not perms_data:
return {}
permissions = {}
try:
if isinstance(perms_data, list) and len(perms_data) > 2:
sections = perms_data[2] if len(perms_data) > 2 else []
if isinstance(sections, list):
for section in sections:
if not isinstance(section, list):
continue
for perm_group in section:
if not isinstance(perm_group, list) or len(perm_group) < 3:
continue
category = None
if isinstance(perm_group[0], str):
category = perm_group[0]
elif isinstance(perm_group[0], list) and len(perm_group[0]) > 0:
category = perm_group[0][0] if isinstance(perm_group[0][0], str) else "Other"
if not category:
category = "Other"
details = []
perm_details = perm_group[2] if len(perm_group) > 2 else []
if isinstance(perm_details, list):
for detail in perm_details:
if isinstance(detail, list) and len(detail) > 1:
if detail[1] and isinstance(detail[1], str):
details.append(detail[1])
elif isinstance(detail, str):
details.append(detail)
if details:
if category in permissions:
permissions[category].extend(details)
else:
permissions[category] = details
if len(sections) > 2:
additional_perms = sections[2] if len(sections) > 2 else []
if isinstance(additional_perms, list):
other_perms = []
for item in additional_perms:
if isinstance(item, list) and len(item) > 1 and isinstance(item[1], str):
other_perms.append(item[1])
if other_perms:
if "Other" in permissions:
permissions["Other"].extend(other_perms)
else:
permissions["Other"] = other_perms
elif isinstance(perms_data, list):
for item in perms_data:
if isinstance(item, list) and len(item) > 2:
category = item[0] if isinstance(item[0], str) else "Other"
details = []
if isinstance(item[2], list):
for detail in item[2]:
if isinstance(detail, list) and len(detail) > 1 and isinstance(detail[1], str):
details.append(detail[1])
if details:
permissions[category] = details
permissions = {k: v for k, v in permissions.items() if v}
except (IndexError, KeyError, TypeError, AttributeError):
pass
return permissions
def nested_lookup(obj: Any, key_list: List) -> Any:
"""Safely navigate nested dictionary/list structure.
Traverses complex nested data structures (mix of dicts and lists) following
a path of keys/indices. Returns None if any step in the path fails.
Args:
obj: Object to navigate (dict, list, or any nested structure)
key_list: List of keys/indices to follow (e.g., [0, 'data', 1, 'title'])
Returns:
Value at the nested location or None if path doesn't exist
Examples:
>>> data = {'users': [{'name': 'John'}, {'name': 'Jane'}]}
>>> nested_lookup(data, ['users', 1, 'name'])
'Jane'
>>> nested_lookup(data, ['users', 5, 'name']) # Index out of range
None
>>> nested_lookup(data, ['invalid', 'path'])
None
"""
current = obj
for key in key_list:
try:
current = current[key]
except (IndexError, KeyError, TypeError):
return None
return current
def format_image_url(url: str, size: str = None) -> str:
"""Format image URL with size parameter.
Google Play Store images can be resized by appending size parameters.
This function adds the appropriate size parameter to get images in desired resolution.
Args:
url: Base image URL from Google Play Store
size: Size parameter - SMALL (512px), MEDIUM (1024px), LARGE (2048px), ORIGINAL (max)
Returns:
Formatted URL with size parameter appended, or None if url is empty
Examples:
>>> format_image_url('https://play-lh.googleusercontent.com/abc123', 'LARGE')
'https://play-lh.googleusercontent.com/abc123=w2048'
>>> format_image_url('https://example.com/image.jpg', 'SMALL')
'https://example.com/image.jpg=w512'
>>> format_image_url('', 'LARGE')
None
"""
if not url:
return None
size_param = Config.get_image_size(size)
return f"{url}={size_param}"
class ElementSpec:
"""Specification for extracting a single field from raw data.
Defines how to extract a specific piece of information from Google Play Store's
complex nested JSON data structures. Each spec contains a navigation path and
optional processing logic.
The extraction process:
1. Navigate through nested data using data_map path
2. Apply post_processor function if specified
3. Return fallback_value if extraction fails
4. Handle asset sizing for image URLs
Attributes:
ds_num: Dataset number (legacy, kept for compatibility)
data_map: List of keys/indices to navigate to the field (e.g., [1, 2, 0, 0])
post_processor: Optional function to process extracted value (e.g., unescape_text)
fallback_value: Value to return if extraction fails (can be another ElementSpec)
assets: Asset size parameter for image URLs
Examples:
# Simple field extraction
title_spec = ElementSpec("raw", [1, 2, 0, 0])
# With post-processing
price_spec = ElementSpec("raw", [1, 2, 57, 0], lambda x: x / 1000000)
# With fallback
version_spec = ElementSpec("raw", [1, 2, 140, 0], fallback_value="Unknown")
"""
def __init__(
self,
ds_num: Optional[int],
data_map: List[int],
post_processor: Callable = None,
fallback_value: Any = None,
assets: str = None,
):
"""Initialize ElementSpec with extraction parameters."""
self.ds_num = ds_num
self.data_map = data_map
self.post_processor = post_processor
self.fallback_value = fallback_value
self.assets = assets
def extract_content(self, source: dict, assets: str = None) -> Any:
"""Extract content from source using data_map.
Performs the actual data extraction by following the navigation path,
applying post-processing, and handling fallbacks.
Args:
source: Source dictionary/list (Google Play Store JSON data)
assets: Override asset size for this extraction (SMALL, MEDIUM, LARGE, ORIGINAL)
Returns:
Extracted and processed value, or fallback_value if extraction fails
Process:
1. Navigate through source data using data_map path
2. Apply post_processor function if available
3. Handle image URL formatting for asset-related fields
4. Return fallback_value if any step fails
Examples:
>>> spec = ElementSpec("raw", [1, 2, 0, 0])
>>> spec.extract_content({'1': {'2': [['App Title']]})
'App Title'
>>> spec.extract_content({'invalid': 'data'})
None # or fallback_value if specified
"""
try:
result = nested_lookup(source, self.data_map)
if self.post_processor is not None:
try:
if hasattr(self.post_processor, '__name__') and 'image' in self.post_processor.__name__:
result = self.post_processor(result, assets or self.assets)
else:
result = self.post_processor(result)
except Exception:
pass
except (KeyError, IndexError, TypeError, AttributeError):
result = None
if result is None and self.fallback_value is not None:
if isinstance(self.fallback_value, ElementSpec):
result = self.fallback_value.extract_content(source, assets)
else:
result = self.fallback_value
return result
class ElementSpecs:
"""Collection of element specifications for all method types.
Central registry of data extraction specifications for all Google Play Store
data types. Each specification defines exactly how to extract specific fields
from the complex nested JSON structures returned by Google's APIs.
Data Categories:
- App: 65+ fields for complete app details (ratings, installs, permissions, etc.)
- Search: Fields for search results (title, developer, price, etc.)
- Review: Fields for user reviews (content, rating, timestamp, etc.)
- Developer: Fields for developer app listings
- Similar: Fields for similar/related apps
- List: Fields for top chart apps (rankings, categories, etc.)
Usage Pattern:
Each category contains ElementSpec objects that define:
- Navigation path through JSON data
- Post-processing functions for data transformation
- Fallback values for missing data
- Asset sizing for images
Example:
>>> app_title = ElementSpecs.App['title'].extract_content(app_data)
>>> search_results = [ElementSpecs.Search['title'].extract_content(item) for item in results]
"""
# App Data Specifications - 65+ fields for complete app analysis
App = {
"title": ElementSpec("raw", [1, 2, 0, 0]),
"description": ElementSpec(
"raw",
[1, 2],
lambda s: (lambda desc_text: unescape_text(desc_text) if desc_text else None)(
nested_lookup(s, [72, 0, 0]) or nested_lookup(s, [72, 0, 1])
),
),
"summary": ElementSpec("raw", [1, 2, 73, 0, 1], unescape_text),
"installs": ElementSpec("raw", [1, 2, 13, 0]),
"minInstalls": ElementSpec("raw", [1, 2, 13, 1]),
"realInstalls": ElementSpec("raw", [1, 2, 13, 2]),
"score": ElementSpec("raw", [1, 2, 51, 0, 1]),
"ratings": ElementSpec("raw", [1, 2, 51, 2, 1]),
"reviews": ElementSpec("raw", [1, 2, 51, 3, 1]),
"histogram": ElementSpec(
"raw",
[1, 2, 51, 1],
lambda container: [
container[1][1],
container[2][1],
container[3][1],
container[4][1],
container[5][1],
],
[0, 0, 0, 0, 0],
),
"price": ElementSpec(
"raw", [1, 2, 57, 0, 0, 0, 0, 1, 0, 0],
lambda price: (price / 1000000) or 0
),
"free": ElementSpec("raw", [1, 2, 57, 0, 0, 0, 0, 1, 0, 0], lambda s: s == 0),
"currency": ElementSpec("raw", [1, 2, 57, 0, 0, 0, 0, 1, 0, 1]),
"sale": ElementSpec("raw", [1, 2, 57, 0, 0, 0, 0, 14, 0, 0], bool, False),
"originalPrice": ElementSpec("raw", [1, 2, 57, 0, 0, 0, 0, 1, 1, 0], lambda price: (price / 1000000) if price else None),
"offersIAP": ElementSpec("raw", [1, 2, 19, 0], bool, False),
"inAppProductPrice": ElementSpec("raw", [1, 2, 19, 0]),
"developer": ElementSpec("raw", [1, 2, 68, 0]),
"developerId": ElementSpec("raw", [1, 2, 68, 1, 4, 2], lambda s: s.split("id=")[1] if s and "id=" in s else None),
"developerEmail": ElementSpec("raw", [1, 2, 69, 1, 0]),
"developerWebsite": ElementSpec("raw", [1, 2, 69, 0, 5, 2]),
"developerAddress": ElementSpec("raw", [1, 2, 69, 4, 2, 0]),
"developerPhone": ElementSpec("raw", [1, 2, 69, 4, 3]),
"privacyPolicy": ElementSpec("raw", [1, 2, 99, 0, 5, 2]),
"genre": ElementSpec("raw", [1, 2, 79, 0, 0, 0]),
"genreId": ElementSpec("raw", [1, 2, 79, 0, 0, 2]),
"categories": ElementSpec("raw", [1, 2, 79, 0, 0, 0], lambda cat: [cat] if cat else [], []),
"icon": ElementSpec("raw", [1, 2, 95, 0, 3, 2]),
"headerImage": ElementSpec("raw", [1, 2, 96, 0, 3, 2]),
"screenshots": ElementSpec("raw", [1, 2, 78, 0], lambda container: [item[3][2] for item in container] if container else [], []),
"video": ElementSpec("raw", [1, 2, 100, 0, 0, 3, 2]),
"videoImage": ElementSpec("raw", [1, 2, 100, 1, 0, 3, 2]),
"contentRating": ElementSpec("raw", [1, 2, 9, 0]),
"contentRatingDescription": ElementSpec("raw", [1, 2, 9, 6, 1], fallback_value=ElementSpec("raw", [1, 2, 9, 2, 1], fallback_value=ElementSpec("raw", [1, 2, 9, 0]))),
"appId": ElementSpec("raw", [1, 2, 1, 0, 0]),
"adSupported": ElementSpec("raw", [1, 2, 48], bool),
"containsAds": ElementSpec("raw", [1, 2, 48], bool, False),
"released": ElementSpec("raw", [1, 2, 10, 0]),
"updated": ElementSpec("raw", [1, 2, 145, 0, 1, 0], fallback_value=ElementSpec("raw", [1, 2, 103, "146", 0, 0], fallback_value=ElementSpec("raw", [1, 2, 145, 0, 0], fallback_value=ElementSpec("raw", [1, 2, 112, "146", 0, 0], fallback_value=ElementSpec("raw", [1, 2, 103, "146", 0, 1,0], fallback_value="Never updated"))))),
"version": ElementSpec("raw", [1, 2, 140, 0, 0, 0], fallback_value=ElementSpec("raw", [1, 2, 103, "141", 0, 0, 0], fallback_value="Varies with device")),
"androidVersion": ElementSpec("raw", [1, 2, 140, 1, 1, 0, 0, 1], fallback_value=ElementSpec("raw", [1, 2, 103, "155", 1, 2], fallback_value=ElementSpec("raw", [1, 2, 112, "141", 1, 1, 0, 0, 0], fallback_value="Varies with device"))),
"permissions": ElementSpec("raw", [1, 2, 74], parse_permissions),
"dataSafety": ElementSpec("raw", [1, 2, 136], lambda data: [item[1] for item in data[1] if item and len(item) > 1] if data and len(data) > 1 and data[1] else []),
"appBundle": ElementSpec("raw", [1, 2, 77, 0]),
"maxandroidapi": ElementSpec("raw", [1, 2, 140, 1, 0, 0, 0], fallback_value=ElementSpec("raw", [1, 2, 103, "141", 1, 0 , 0, 0], fallback_value=ElementSpec("raw", [1, 2, 112, "141", 1, 0, 0, 0], fallback_value="Varies with device"))),
"minandroidapi": ElementSpec("raw", [1, 2, 140, 1, 1, 0, 0, 0], fallback_value=ElementSpec("raw", [1, 2, 103, "141", 1, 1, 0, 0, 0], fallback_value=ElementSpec("raw", [1, 2, 112, "141", 1, 1, 0, 0, 0], fallback_value="Varies with device"))),
"whatsNew": ElementSpec("raw", [1, 2, 144, 1, 1], lambda x: [line.strip() for line in html.unescape(x).split('
') if line.strip()] if x else []),
"available": ElementSpec("raw", [1, 2, 18, 0], bool, False),
"url": ElementSpec("raw", [1, 2, 1, 0, 0], lambda app_id: f"https://play.google.com/store/apps/details?id={app_id}" if app_id else None),
}
# Search Results Specifications - Fields for app search results
Search = {
"title": ElementSpec("raw", [2]),
"appId": ElementSpec("raw", [12, 0]),
"icon": ElementSpec("raw", [1, 1, 0, 3, 2]),
"developer": ElementSpec("raw", [4, 0, 0, 0]),
"currency": ElementSpec("raw", [7, 0, 3, 2, 1, 0, 1]),
"price": ElementSpec("raw", [7, 0, 3, 2, 1, 0, 0], lambda price: (price / 1000000) if price else 0),
"free": ElementSpec("raw", [7, 0, 3, 2, 1, 0, 0], lambda s: s == 0),
"summary": ElementSpec("raw", [4, 1, 1, 1, 1], unescape_text),
"scoreText": ElementSpec("raw", [6, 0, 2, 1, 0]),
"score": ElementSpec("raw", [6, 0, 2, 1, 1]),
"url": ElementSpec("raw", [12, 0], lambda app_id: f"https://play.google.com/store/apps/details?id={app_id}" if app_id else None),
}
# Review Data Specifications - Fields for user reviews
Review = {
"reviewId": ElementSpec("raw", [0]),
"userName": ElementSpec("raw", [1, 0]),
"userImage": ElementSpec("raw", [1, 1, 3, 2]),
"content": ElementSpec("raw", [4], unescape_text),
"score": ElementSpec("raw", [2]),
"thumbsUpCount": ElementSpec("raw", [6]),
"at": ElementSpec("raw", [5, 0], lambda timestamp: datetime.fromtimestamp(timestamp).isoformat() if timestamp else None),
"appVersion": ElementSpec("raw", [10]),
}
# Developer App Specifications - Fields for developer's app listings
Developer = {
"appId": ElementSpec("raw", [0, 0]),
"title": ElementSpec("raw", [3]),
"icon": ElementSpec("raw", [1, 3, 2]),
"developer": ElementSpec("raw", [14]),
"description": ElementSpec("raw", [13, 1], unescape_text),
"score": ElementSpec("raw", [4, 1]),
"scoreText": ElementSpec("raw", [4, 0]),
"price": ElementSpec("raw", [8, 1, 0, 0], lambda price: (price / 1000000) if price else 0),
"currency": ElementSpec("raw", [8, 1, 0, 1]),
"free": ElementSpec("raw", [8, 1, 0, 0], lambda s: s == 0),
"url": ElementSpec("raw", [10, 4, 2], lambda path: f"https://play.google.com{path}" if path else None),
}
# Similar Apps Specifications - Fields for related/similar apps
Similar = {
"appId": ElementSpec("raw", [0, 0]),
"title": ElementSpec("raw", [3]),
"icon": ElementSpec("raw", [1, 3, 2]),
"developer": ElementSpec("raw", [14]),
"description": ElementSpec("raw", [13, 1], unescape_text),
"score": ElementSpec("raw", [4, 1]),
"scoreText": ElementSpec("raw", [4, 0]),
"price": ElementSpec("raw", [8, 1, 0, 0], lambda price: (price / 1000000) if price else 0),
"currency": ElementSpec("raw", [8, 1, 0, 1]),
"free": ElementSpec("raw", [8, 1, 0, 0], lambda s: s == 0),
"url": ElementSpec("raw", [10, 4, 2], lambda path: f"https://play.google.com{path}" if path else None),
}
# Top Charts Specifications - Fields for ranked app lists
List = {
"title": ElementSpec("raw", [0, 3]),
"appId": ElementSpec("raw", [0, 0, 0]),
"icon": ElementSpec("raw", [0, 1, 3, 2]),
"screenshots": ElementSpec("raw", [0, 2], lambda container: [s[3][2] for s in container if s and len(s) > 3] if container else [], []),
"developer": ElementSpec("raw", [0, 14]),
"genre": ElementSpec("raw", [0, 5]),
"installs": ElementSpec("raw", [0, 15]),
"currency": ElementSpec("raw", [0, 8, 1, 0, 1]),
"price": ElementSpec("raw", [0, 8, 1, 0, 0], lambda price: (price / 1000000) if price else 0),
"free": ElementSpec("raw", [0, 8, 1, 0, 0], lambda s: s == 0),
"description": ElementSpec("raw", [0, 13, 1], unescape_text),
"scoreText": ElementSpec("raw", [0, 4, 0]),
"score": ElementSpec("raw", [0, 4, 1]),
"url": ElementSpec("raw", [0, 10, 4, 2], lambda path: f"https://play.google.com{path}" if path else None),
}
================================================
FILE: gplay_scraper/utils/__init__.py
================================================
from .helpers import *
__all__ = [
'nested_lookup', 'unescape_text', 'extract_categories', 'get_categories',
'parse_release_date', 'calculate_app_age', 'parse_installs_string',
'calculate_daily_installs', 'calculate_monthly_installs', 'clean_json_string'
]
================================================
FILE: gplay_scraper/utils/constants.py
================================================
# Review sort options
SORT_NAMES = {
'RELEVANT': 1, # Most relevant reviews
'NEWEST': 2, # Newest reviews first
'RATING': 3 # Sorted by rating
}
# List collection types
CLUSTER_NAMES = {
'TOP_FREE': 'topselling_free', # Top free apps
'TOP_PAID': 'topselling_paid', # Top paid apps
'TOP_GROSSING': 'topgrossing' # Top grossing apps
}
# Phone country code mappings
PHONE_PREFIXES = [
(1201, 'us', 'united states'),
(1202, 'us', 'united states'),
(1203, 'us', 'united states'),
(1204, 'ca', 'canada'),
(1205, 'us', 'united states'),
(1206, 'us', 'united states'),
(1207, 'us', 'united states'),
(1208, 'us', 'united states'),
(1209, 'us', 'united states'),
(1210, 'us', 'united states'),
(1212, 'us', 'united states'),
(1213, 'us', 'united states'),
(1214, 'us', 'united states'),
(1215, 'us', 'united states'),
(1216, 'us', 'united states'),
(1217, 'us', 'united states'),
(1218, 'us', 'united states'),
(1219, 'us', 'united states'),
(1224, 'us', 'united states'),
(1225, 'us', 'united states'),
(1226, 'ca', 'canada'),
(1228, 'us', 'united states'),
(1229, 'us', 'united states'),
(1231, 'us', 'united states'),
(1234, 'us', 'united states'),
(1236, 'ca', 'canada'),
(1239, 'us', 'united states'),
(1240, 'us', 'united states'),
(1242, 'bs', 'bahamas'),
(1246, 'bb', 'barbados'),
(1248, 'us', 'united states'),
(1249, 'ca', 'canada'),
(1250, 'ca', 'canada'),
(1251, 'us', 'united states'),
(1252, 'us', 'united states'),
(1253, 'us', 'united states'),
(1254, 'us', 'united states'),
(1256, 'us', 'united states'),
(1260, 'us', 'united states'),
(1262, 'us', 'united states'),
(1264, 'ai', 'anguilla'),
(1267, 'us', 'united states'),
(1268, 'ag', 'antigua and barbuda'),
(1269, 'us', 'united states'),
(1270, 'us', 'united states'),
(1272, 'us', 'united states'),
(1274, 'us', 'united states'),
(1276, 'us', 'united states'),
(1281, 'us', 'united states'),
(1284, 'vg', 'british virgin islands'),
(1289, 'ca', 'canada'),
(1301, 'us', 'united states'),
(1302, 'us', 'united states'),
(1303, 'us', 'united states'),
(1304, 'us', 'united states'),
(1305, 'us', 'united states'),
(1306, 'ca', 'canada'),
(1307, 'us', 'united states'),
(1308, 'us', 'united states'),
(1309, 'us', 'united states'),
(1310, 'us', 'united states'),
(1312, 'us', 'united states'),
(1313, 'us', 'united states'),
(1314, 'us', 'united states'),
(1315, 'us', 'united states'),
(1316, 'us', 'united states'),
(1317, 'us', 'united states'),
(1318, 'us', 'united states'),
(1319, 'us', 'united states'),
(1320, 'us', 'united states'),
(1321, 'us', 'united states'),
(1323, 'us', 'united states'),
(1325, 'us', 'united states'),
(1330, 'us', 'united states'),
(1331, 'us', 'united states'),
(1334, 'us', 'united states'),
(1336, 'us', 'united states'),
(1337, 'us', 'united states'),
(1339, 'us', 'united states'),
(1340, 'vi', 'u.s. virgin islands'),
(1343, 'ca', 'canada'),
(1345, 'ky', 'cayman islands'),
(1346, 'us', 'united states'),
(1347, 'us', 'united states'),
(1351, 'us', 'united states'),
(1352, 'us', 'united states'),
(1360, 'us', 'united states'),
(1361, 'us', 'united states'),
(1364, 'us', 'united states'),
(1365, 'ca', 'canada'),
(1385, 'us', 'united states'),
(1386, 'us', 'united states'),
(1401, 'us', 'united states'),
(1402, 'us', 'united states'),
(1403, 'ca', 'canada'),
(1404, 'us', 'united states'),
(1405, 'us', 'united states'),
(1406, 'us', 'united states'),
(1407, 'us', 'united states'),
(1408, 'us', 'united states'),
(1409, 'us', 'united states'),
(1410, 'us', 'united states'),
(1412, 'us', 'united states'),
(1413, 'us', 'united states'),
(1414, 'us', 'united states'),
(1415, 'us', 'united states'),
(1416, 'ca', 'canada'),
(1417, 'us', 'united states'),
(1418, 'ca', 'canada'),
(1419, 'us', 'united states'),
(1423, 'us', 'united states'),
(1424, 'us', 'united states'),
(1425, 'us', 'united states'),
(1430, 'us', 'united states'),
(1431, 'ca', 'canada'),
(1432, 'us', 'united states'),
(1434, 'us', 'united states'),
(1435, 'us', 'united states'),
(1437, 'ca', 'canada'),
(1438, 'ca', 'canada'),
(1440, 'us', 'united states'),
(1441, 'bm', 'bermuda'),
(1442, 'us', 'united states'),
(1443, 'us', 'united states'),
(1450, 'ca', 'canada'),
(1457, 'ca', 'canada'),
(1458, 'us', 'united states'),
(1469, 'us', 'united states'),
(1470, 'us', 'united states'),
(1473, 'gd', 'grenada'),
(1475, 'us', 'united states'),
(1478, 'us', 'united states'),
(1479, 'us', 'united states'),
(1480, 'us', 'united states'),
(1484, 'us', 'united states'),
(1500, 'us', 'united states'),
(1501, 'us', 'united states'),
(1502, 'us', 'united states'),
(1503, 'us', 'united states'),
(1504, 'us', 'united states'),
(1505, 'us', 'united states'),
(1506, 'ca', 'canada'),
(1507, 'us', 'united states'),
(1508, 'us', 'united states'),
(1509, 'us', 'united states'),
(1510, 'us', 'united states'),
(1512, 'us', 'united states'),
(1513, 'us', 'united states'),
(1514, 'ca', 'canada'),
(1515, 'us', 'united states'),
(1516, 'us', 'united states'),
(1517, 'us', 'united states'),
(1518, 'us', 'united states'),
(1519, 'ca', 'canada'),
(1520, 'us', 'united states'),
(1530, 'us', 'united states'),
(1531, 'us', 'united states'),
(1533, 'us', 'united states'),
(1534, 'us', 'united states'),
(1539, 'us', 'united states'),
(1540, 'us', 'united states'),
(1541, 'us', 'united states'),
(1544, 'us', 'united states'),
(1551, 'us', 'united states'),
(1559, 'us', 'united states'),
(1561, 'us', 'united states'),
(1562, 'us', 'united states'),
(1563, 'us', 'united states'),
(1566, 'us', 'united states'),
(1567, 'us', 'united states'),
(1570, 'us', 'united states'),
(1571, 'us', 'united states'),
(1573, 'us', 'united states'),
(1574, 'us', 'united states'),
(1575, 'us', 'united states'),
(1577, 'us', 'united states'),
(1579, 'ca', 'canada'),
(1580, 'us', 'united states'),
(1581, 'ca', 'canada'),
(1585, 'us', 'united states'),
(1586, 'us', 'united states'),
(1587, 'ca', 'canada'),
(1600, 'ca', 'canada'),
(1601, 'us', 'united states'),
(1602, 'us', 'united states'),
(1603, 'us', 'united states'),
(1604, 'ca', 'canada'),
(1605, 'us', 'united states'),
(1606, 'us', 'united states'),
(1607, 'us', 'united states'),
(1608, 'us', 'united states'),
(1609, 'us', 'united states'),
(1610, 'us', 'united states'),
(1612, 'us', 'united states'),
(1613, 'ca', 'canada'),
(1614, 'us', 'united states'),
(1615, 'us', 'united states'),
(1616, 'us', 'united states'),
(1617, 'us', 'united states'),
(1618, 'us', 'united states'),
(1619, 'us', 'united states'),
(1620, 'us', 'united states'),
(1623, 'us', 'united states'),
(1626, 'us', 'united states'),
(1628, 'us', 'united states'),
(1629, 'us', 'united states'),
(1630, 'us', 'united states'),
(1631, 'us', 'united states'),
(1636, 'us', 'united states'),
(1639, 'ca', 'canada'),
(1641, 'us', 'united states'),
(1646, 'us', 'united states'),
(1647, 'ca', 'canada'),
(1649, 'tc', 'turks and caicos islands'),
(1650, 'us', 'united states'),
(1651, 'us', 'united states'),
(1657, 'us', 'united states'),
(1660, 'us', 'united states'),
(1661, 'us', 'united states'),
(1662, 'us', 'united states'),
(1664, 'ms', 'montserrat'),
(1667, 'us', 'united states'),
(1669, 'us', 'united states'),
(1670, 'mp', 'northern mariana islands'),
(1671, 'gu', 'guam'),
(1678, 'us', 'united states'),
(1681, 'us', 'united states'),
(1682, 'us', 'united states'),
(1684, 'as', 'american samoa'),
(1700, 'us', 'united states'),
(1701, 'us', 'united states'),
(1702, 'us', 'united states'),
(1703, 'us', 'united states'),
(1704, 'us', 'united states'),
(1705, 'ca', 'canada'),
(1706, 'us', 'united states'),
(1707, 'us', 'united states'),
(1708, 'us', 'united states'),
(1709, 'ca', 'canada'),
(1710, 'us', 'united states'),
(1712, 'us', 'united states'),
(1713, 'us', 'united states'),
(1714, 'us', 'united states'),
(1715, 'us', 'united states'),
(1716, 'us', 'united states'),
(1717, 'us', 'united states'),
(1718, 'us', 'united states'),
(1719, 'us', 'united states'),
(1720, 'us', 'united states'),
(1721, 'sx', 'sint maarten'),
(1724, 'us', 'united states'),
(1725, 'us', 'united states'),
(1727, 'us', 'united states'),
(1731, 'us', 'united states'),
(1732, 'us', 'united states'),
(1734, 'us', 'united states'),
(1737, 'us', 'united states'),
(1740, 'us', 'united states'),
(1747, 'us', 'united states'),
(1754, 'us', 'united states'),
(1757, 'us', 'united states'),
(1758, 'lc', 'saint lucia'),
(1760, 'us', 'united states'),
(1762, 'us', 'united states'),
(1763, 'us', 'united states'),
(1765, 'us', 'united states'),
(1767, 'dm', 'dominica'),
(1769, 'us', 'united states'),
(1770, 'us', 'united states'),
(1772, 'us', 'united states'),
(1773, 'us', 'united states'),
(1774, 'us', 'united states'),
(1775, 'us', 'united states'),
(1778, 'ca', 'canada'),
(1779, 'us', 'united states'),
(1780, 'ca', 'canada'),
(1781, 'us', 'united states'),
(1782, 'ca', 'canada'),
(1784, 'vc', 'saint vincent and the grenadines'),
(1785, 'us', 'united states'),
(1786, 'us', 'united states'),
(1787, 'pr', 'puerto rico'),
(1800, 'us', 'united states'),
(1801, 'us', 'united states'),
(1802, 'us', 'united states'),
(1803, 'us', 'united states'),
(1804, 'us', 'united states'),
(1805, 'us', 'united states'),
(1806, 'us', 'united states'),
(1807, 'ca', 'canada'),
(1808, 'us', 'united states'),
(1809, 'do', 'dominican republic'),
(1810, 'us', 'united states'),
(1812, 'us', 'united states'),
(1813, 'us', 'united states'),
(1814, 'us', 'united states'),
(1815, 'us', 'united states'),
(1816, 'us', 'united states'),
(1817, 'us', 'united states'),
(1818, 'us', 'united states'),
(1819, 'ca', 'canada'),
(1825, 'ca', 'canada'),
(1828, 'us', 'united states'),
(1829, 'do', 'dominican republic'),
(1830, 'us', 'united states'),
(1831, 'us', 'united states'),
(1832, 'us', 'united states'),
(1843, 'us', 'united states'),
(1844, 'us', 'united states'),
(1845, 'us', 'united states'),
(1847, 'us', 'united states'),
(1848, 'us', 'united states'),
(1849, 'do', 'dominican republic'),
(1850, 'us', 'united states'),
(1855, 'us', 'united states'),
(1856, 'us', 'united states'),
(1857, 'us', 'united states'),
(1858, 'us', 'united states'),
(1859, 'us', 'united states'),
(1860, 'us', 'united states'),
(1862, 'us', 'united states'),
(1863, 'us', 'united states'),
(1864, 'us', 'united states'),
(1865, 'us', 'united states'),
(1866, 'us', 'united states'),
(1867, 'ca', 'canada'),
(1868, 'tt', 'trinidad and tobago'),
(1869, 'kn', 'saint kitts and nevis'),
(1870, 'us', 'united states'),
(1872, 'us', 'united states'),
(1873, 'ca', 'canada'),
(1876, 'jm', 'jamaica'),
(1877, 'us', 'united states'),
(1878, 'us', 'united states'),
(1888, 'us', 'united states'),
(1900, 'us', 'united states'),
(1901, 'us', 'united states'),
(1902, 'ca', 'canada'),
(1903, 'us', 'united states'),
(1904, 'us', 'united states'),
(1905, 'ca', 'canada'),
(1906, 'us', 'united states'),
(1907, 'us', 'united states'),
(1908, 'us', 'united states'),
(1909, 'us', 'united states'),
(1910, 'us', 'united states'),
(1912, 'us', 'united states'),
(1913, 'us', 'united states'),
(1914, 'us', 'united states'),
(1915, 'us', 'united states'),
(1916, 'us', 'united states'),
(1917, 'us', 'united states'),
(1918, 'us', 'united states'),
(1919, 'us', 'united states'),
(1920, 'us', 'united states'),
(1925, 'us', 'united states'),
(1928, 'us', 'united states'),
(1929, 'us', 'united states'),
(1930, 'us', 'united states'),
(1931, 'us', 'united states'),
(1935, 'us', 'united states'),
(1936, 'us', 'united states'),
(1937, 'us', 'united states'),
(1938, 'us', 'united states'),
(1939, 'pr', 'puerto rico'),
(1940, 'us', 'united states'),
(1941, 'us', 'united states'),
(1947, 'us', 'united states'),
(1949, 'us', 'united states'),
(1951, 'us', 'united states'),
(1952, 'us', 'united states'),
(1954, 'us', 'united states'),
(1956, 'us', 'united states'),
(1959, 'us', 'united states'),
(1970, 'us', 'united states'),
(1971, 'us', 'united states'),
(1972, 'us', 'united states'),
(1973, 'us', 'united states'),
(1978, 'us', 'united states'),
(1979, 'us', 'united states'),
(1980, 'us', 'united states'),
(1984, 'us', 'united states'),
(1985, 'us', 'united states'),
(1989, 'us', 'united states'),
(20, 'eg', 'egypt'),
(211, 'ss', 'south sudan'),
(212, 'ma', 'morocco'),
(213, 'dz', 'algeria'),
(216, 'tn', 'tunisia'),
(218, 'ly', 'libya'),
(220, 'gm', 'gambia'),
(221, 'sn', 'senegal'),
(222, 'mr', 'mauritania'),
(223, 'ml', 'mali'),
(224, 'gn', 'guinea'),
(225, 'ci', 'ivory coast'),
(226, 'bf', 'burkina faso'),
(227, 'ne', 'niger'),
(228, 'tg', 'togo'),
(229, 'bj', 'benin'),
(230, 'mu', 'mauritius'),
(231, 'lr', 'liberia'),
(232, 'sl', 'sierra leone'),
(233, 'gh', 'ghana'),
(234, 'ng', 'nigeria'),
(235, 'td', 'chad'),
(236, 'cf', 'central african republic'),
(237, 'cm', 'cameroon'),
(238, 'cv', 'cape verde'),
(239, 'st', 'sao tome and principe'),
(240, 'gq', 'equatorial guinea'),
(241, 'ga', 'gabon'),
(242, 'cg', 'republic of the congo'),
(243, 'cd', 'democratic republic of the congo'),
(244, 'ao', 'angola'),
(245, 'gw', 'guinea-bissau'),
(246, 'io', 'british indian ocean territory'),
(247, 'ac', 'ascension island'),
(248, 'sc', 'seychelles'),
(249, 'sd', 'sudan'),
(250, 'rw', 'rwanda'),
(251, 'et', 'ethiopia'),
(252, 'so', 'somalia'),
(253, 'dj', 'djibouti'),
(254, 'ke', 'kenya'),
(255, 'tz', 'tanzania'),
(256, 'ug', 'uganda'),
(257, 'bi', 'burundi'),
(258, 'mz', 'mozambique'),
(260, 'zm', 'zambia'),
(261, 'mg', 'madagascar'),
(262, 're', 'réunion'),
(262269, 'yt', 'mayotte'),
(262639, 'yt', 'mayotte'),
(263, 'zw', 'zimbabwe'),
(264, 'na', 'namibia'),
(265, 'mw', 'malawi'),
(266, 'ls', 'lesotho'),
(267, 'bw', 'botswana'),
(268, 'sz', 'eswatini'),
(269, 'km', 'comoros'),
(27, 'za', 'south africa'),
(290, 'sh', 'saint helena'),
(291, 'er', 'eritrea'),
(297, 'aw', 'aruba'),
(298, 'fo', 'faroe islands'),
(299, 'gl', 'greenland'),
(30, 'gr', 'greece'),
(31, 'nl', 'netherlands'),
(32, 'be', 'belgium'),
(33, 'fr', 'france'),
(34, 'es', 'spain'),
(350, 'gi', 'gibraltar'),
(351, 'pt', 'portugal'),
(352, 'lu', 'luxembourg'),
(353, 'ie', 'ireland'),
(354, 'is', 'iceland'),
(355, 'al', 'albania'),
(356, 'mt', 'malta'),
(357, 'cy', 'cyprus'),
(358, 'fi', 'finland'),
(35818, 'ax', 'åland islands'),
(359, 'bg', 'bulgaria'),
(36, 'hu', 'hungary'),
(370, 'lt', 'lithuania'),
(371, 'lv', 'latvia'),
(372, 'ee', 'estonia'),
(373, 'md', 'moldova'),
(374, 'am', 'armenia'),
(375, 'by', 'belarus'),
(376, 'ad', 'andorra'),
(377, 'mc', 'monaco'),
(378, 'sm', 'san marino'),
(379, 'va', 'vatican city'),
(380, 'ua', 'ukraine'),
(381, 'rs', 'serbia'),
(382, 'me', 'montenegro'),
(385, 'hr', 'croatia'),
(386, 'si', 'slovenia'),
(387, 'ba', 'bosnia and herzegovina'),
(389, 'mk', 'north macedonia'),
(39, 'it', 'italy'),
(40, 'ro', 'romania'),
(41, 'ch', 'switzerland'),
(420, 'cz', 'czech republic'),
(421, 'sk', 'slovakia'),
(423, 'li', 'liechtenstein'),
(43, 'at', 'austria'),
(441481, 'gg', 'guernsey'),
(441624, 'im', 'isle of man'),
(441534, 'je', 'jersey'),
(44, 'gb', 'united kingdom'),
(45, 'dk', 'denmark'),
(46, 'se', 'sweden'),
(47, 'no', 'norway'),
(4779, 'sj', 'svalbard and jan mayen'),
(48, 'pl', 'poland'),
(49, 'de', 'germany'),
(500, 'fk', 'falkland islands'),
(501, 'bz', 'belize'),
(502, 'gt', 'guatemala'),
(503, 'sv', 'el salvador'),
(504, 'hn', 'honduras'),
(505, 'ni', 'nicaragua'),
(506, 'cr', 'costa rica'),
(507, 'pa', 'panama'),
(508, 'pm', 'saint pierre and miquelon'),
(509, 'ht', 'haiti'),
(51, 'pe', 'peru'),
(52, 'mx', 'mexico'),
(53, 'cu', 'cuba'),
(54, 'ar', 'argentina'),
(55, 'br', 'brazil'),
(56, 'cl', 'chile'),
(57, 'co', 'colombia'),
(58, 've', 'venezuela'),
(590, 'gp', 'guadeloupe'),
(591, 'bo', 'bolivia'),
(592, 'gy', 'guyana'),
(593, 'ec', 'ecuador'),
(594, 'gf', 'french guiana'),
(595, 'py', 'paraguay'),
(596, 'mq', 'martinique'),
(597, 'sr', 'suriname'),
(598, 'uy', 'uruguay'),
(5993, 'bq', 'bonaire, sint eustatius and saba'),
(5994, 'bq', 'bonaire, sint eustatius and saba'),
(5997, 'bq', 'bonaire, sint eustatius and saba'),
(5999, 'cw', 'curaçao'),
(60, 'my', 'malaysia'),
(61, 'au', 'australia'),
(6189164, 'cx', 'christmas island'),
(6189162, 'cc', 'cocos (keeling) islands'),
(62, 'id', 'indonesia'),
(63, 'ph', 'philippines'),
(64, 'nz', 'new zealand'),
(65, 'sg', 'singapore'),
(66, 'th', 'thailand'),
(670, 'tl', 'timor-leste'),
(6721, 'aq', 'antarctica'),
(6723, 'nf', 'norfolk island'),
(673, 'bn', 'brunei'),
(674, 'nr', 'nauru'),
(675, 'pg', 'papua new guinea'),
(676, 'to', 'tonga'),
(677, 'sb', 'solomon islands'),
(678, 'vu', 'vanuatu'),
(679, 'fj', 'fiji'),
(680, 'pw', 'palau'),
(681, 'wf', 'wallis and futuna'),
(682, 'ck', 'cook islands'),
(683, 'nu', 'niue'),
(685, 'ws', 'samoa'),
(686, 'ki', 'kiribati'),
(687, 'nc', 'new caledonia'),
(688, 'tv', 'tuvalu'),
(689, 'pf', 'french polynesia'),
(690, 'tk', 'tokelau'),
(691, 'fm', 'micronesia'),
(692, 'mh', 'marshall islands'),
(7, 'ru', 'russia'),
(76, 'kz', 'kazakhstan'),
(77, 'kz', 'kazakhstan'),
(800, 'xt', 'international toll-free'),
(808, 'xs', 'shared-cost service'),
(81, 'jp', 'japan'),
(82, 'kr', 'south korea'),
(84, 'vn', 'vietnam'),
(850, 'kp', 'north korea'),
(852, 'hk', 'hong kong'),
(853, 'mo', 'macao'),
(855, 'kh', 'cambodia'),
(856, 'la', 'laos'),
(86, 'cn', 'china'),
(870, 'xn', 'inmarsat'),
(878, 'xp', 'universal personal telecommunications'),
(880, 'bd', 'bangladesh'),
(881, 'xg', 'global mobile satellite system'),
(882, 'xv', 'international networks'),
(883, 'xv', 'international networks'),
(886, 'tw', 'taiwan'),
(90, 'tr', 'turkey'),
(91, 'in', 'india'),
(92, 'pk', 'pakistan'),
(93, 'af', 'afghanistan'),
(94, 'lk', 'sri lanka'),
(95, 'mm', 'myanmar'),
(960, 'mv', 'maldives'),
(961, 'lb', 'lebanon'),
(962, 'jo', 'jordan'),
(963, 'sy', 'syria'),
(964, 'iq', 'iraq'),
(965, 'kw', 'kuwait'),
(966, 'sa', 'saudi arabia'),
(967, 'ye', 'yemen'),
(968, 'om', 'oman'),
(970, 'ps', 'palestine'),
(971, 'ae', 'united arab emirates'),
(972, 'il', 'israel'),
(973, 'bh', 'bahrain'),
(974, 'qa', 'qatar'),
(975, 'bt', 'bhutan'),
(976, 'mn', 'mongolia'),
(977, 'np', 'nepal'),
(98, 'ir', 'iran'),
(992, 'tj', 'tajikistan'),
(993, 'tm', 'turkmenistan'),
(994, 'az', 'azerbaijan'),
(995, 'ge', 'georgia'),
(996, 'kg', 'kyrgyzstan'),
(998, 'uz', 'uzbekistan')
]
================================================
FILE: gplay_scraper/utils/error_handling.py
================================================
"""Unified error handling decorators for all gplay_scraper methods."""
import time
import logging
import json
from functools import wraps
from ..config import Config
from ..exceptions import AppNotFoundError, NetworkError, DataParsingError, RateLimitError, InvalidAppIdError
logger = logging.getLogger(__name__)
def retry_on_not_found(max_retries=Config.DEFAULT_RETRY_COUNT, delay=1.0):
"""Decorator to retry methods on AppNotFoundError with all HTTP clients.
This decorator implements automatic retry logic when apps are not found,
cycling through different HTTP clients to overcome potential blocking.
Args:
max_retries: Maximum number of retry attempts (default: 3)
delay: Delay in seconds between retries (default: 1.0)
Returns:
Decorated function with retry logic
Example:
@retry_on_not_found(max_retries=5, delay=2.0)
def fetch_app_data(self, app_id):
# Method implementation
pass
"""
def decorator(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
for attempt in range(max_retries):
try:
return func(self, *args, **kwargs)
except AppNotFoundError as e:
if attempt < max_retries - 1:
logger.warning(f"Attempt {attempt + 1} failed: {e}. Retrying in {delay}s...")
time.sleep(delay)
# Switch to next HTTP client for retry
if hasattr(self, 'scraper') and hasattr(self.scraper, 'http_client'):
self.scraper.http_client._try_next_client()
continue
except Exception as e:
# Re-raise non-recoverable exceptions immediately
raise e
logger.error(f"All {max_retries} attempts failed. Skipping.")
return None
return wrapper
return decorator
def handle_network_errors(return_empty=False):
"""Decorator to handle network errors gracefully.
Catches network-related exceptions and provides graceful degradation
instead of crashing the application.
Args:
return_empty: If True, return empty list/dict on errors instead of None
Returns:
Decorated function with network error handling
Example:
@handle_network_errors(return_empty=True)
def fetch_search_results(self, query):
# Method implementation that may fail due to network issues
pass
"""
def decorator(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
try:
return func(self, *args, **kwargs)
except NetworkError as e:
logger.warning(f"Network error in {func.__name__}: {e}")
return [] if return_empty else None
except Exception as e:
logger.error(f"Unexpected error in {func.__name__}: {e}")
return [] if return_empty else None
return wrapper
return decorator
def handle_parsing_errors(return_empty=False):
"""Decorator to handle data parsing errors gracefully.
Catches JSON parsing and data extraction errors that may occur when
Google Play Store changes their data structure.
Args:
return_empty: If True, return empty list/dict on errors instead of None
Returns:
Decorated function with parsing error handling
Example:
@handle_parsing_errors(return_empty=True)
def parse_app_data(self, raw_data):
# Method implementation that may fail due to data format changes
pass
"""
def decorator(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
try:
return func(self, *args, **kwargs)
except DataParsingError as e:
logger.warning(f"Parsing error in {func.__name__}: {e}")
return [] if return_empty else None
except Exception as e:
logger.error(f"Unexpected error in {func.__name__}: {e}")
return [] if return_empty else None
return wrapper
return decorator
def handle_rate_limit():
"""Decorator to handle rate limiting with exponential backoff.
Implements exponential backoff strategy when rate limits are encountered,
gradually increasing delay between retries to avoid overwhelming the server.
Returns:
Decorated function with rate limit handling
Example:
@handle_rate_limit()
def make_api_request(self, endpoint):
# Method implementation that may trigger rate limits
pass
Note:
Uses exponential backoff: 1s, 2s, 4s, 8s, etc.
"""
def decorator(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
max_attempts = Config.DEFAULT_RETRY_COUNT
base_delay = Config.RATE_LIMIT_DELAY
for attempt in range(max_attempts):
try:
return func(self, *args, **kwargs)
except RateLimitError as e:
if attempt < max_attempts - 1:
# Exponential backoff: 1s, 2s, 4s, 8s...
delay = base_delay * (2 ** attempt)
logger.warning(f"Rate limited. Waiting {delay}s before retry...")
time.sleep(delay)
continue
logger.error(f"Rate limit exceeded after {max_attempts} attempts")
return None
except Exception as e:
# Re-raise non-rate-limit exceptions
raise e
return wrapper
return decorator
def validate_inputs():
"""Decorator to validate method inputs.
Performs basic input validation to ensure app IDs, queries, and other
parameters are valid before processing.
Returns:
Decorated function with input validation
Raises:
InvalidAppIdError: When input validation fails
Example:
@validate_inputs()
def fetch_app_details(self, app_id):
# Method implementation with validated inputs
pass
Note:
Validates that first argument is non-empty string
"""
def decorator(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
try:
# Validate first argument (usually app_id, query, etc.)
if args and not args[0]:
raise InvalidAppIdError("Input cannot be empty")
if args and not isinstance(args[0], str):
raise InvalidAppIdError("Input must be a string")
return func(self, *args, **kwargs)
except InvalidAppIdError:
# Re-raise validation errors as-is
raise
except Exception as e:
logger.error(f"Input validation error in {func.__name__}: {e}")
raise InvalidAppIdError(f"Invalid input: {e}")
return wrapper
return decorator
def safe_print():
"""Decorator to handle Unicode errors in print methods.
Handles Unicode encoding issues when printing data containing
special characters from different languages.
Returns:
Decorated function with Unicode-safe printing
Example:
@safe_print()
def print_app_data(self, app_id):
# Method implementation that prints data with Unicode characters
pass
Note:
Falls back to ASCII encoding if Unicode printing fails
"""
def decorator(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
try:
return func(self, *args, **kwargs)
except UnicodeEncodeError:
# Fallback: get data and print with ASCII encoding
data = getattr(self, func.__name__.replace('print', 'analyze'))(*args, **kwargs)
if data:
print(json.dumps(data, indent=2, ensure_ascii=True))
else:
print("No data available")
except Exception as e:
print(f"Error: {e}")
return wrapper
return decorator
def comprehensive_error_handler(return_empty=False):
"""Comprehensive decorator combining all error handling strategies.
This decorator provides unified error handling for all scraper methods,
combining input validation, retry logic, network error handling,
and graceful degradation in a single decorator.
Args:
return_empty: If True, return empty list/dict on errors instead of None
Returns:
Decorated function with comprehensive error handling
Example:
@comprehensive_error_handler(return_empty=True)
def scrape_app_data(self, app_id):
# Method implementation with full error protection
pass
Features:
- Input validation
- Automatic retries with HTTP client fallback
- Network and parsing error handling
- Rate limit management
- Graceful degradation
"""
def decorator(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
try:
# Input validation
if args and not args[0]:
raise InvalidAppIdError("Input cannot be empty")
if args and not isinstance(args[0], str):
raise InvalidAppIdError("Input must be a string")
# Retry logic with HTTP client fallback
max_retries = Config.DEFAULT_RETRY_COUNT
for attempt in range(max_retries):
try:
return func(self, *args, **kwargs)
except AppNotFoundError as e:
if attempt < max_retries - 1:
logger.warning(f"Attempt {attempt + 1} failed: {e}. Retrying...")
time.sleep(Config.RATE_LIMIT_DELAY)
# Switch to next HTTP client for retry
if hasattr(self, 'scraper') and hasattr(self.scraper, 'http_client'):
self.scraper.http_client._try_next_client()
continue
logger.error(f"All {max_retries} attempts failed")
return [] if return_empty else None
except (NetworkError, DataParsingError, RateLimitError) as e:
# Handle recoverable errors gracefully
logger.warning(f"Recoverable error in {func.__name__}: {e}")
return [] if return_empty else None
except Exception as e:
# Handle unexpected errors
logger.error(f"Unexpected error in {func.__name__}: {e}")
return [] if return_empty else None
except InvalidAppIdError:
# Re-raise validation errors
raise
except Exception as e:
# Handle critical errors
logger.error(f"Critical error in {func.__name__}: {e}")
return [] if return_empty else None
return wrapper
return decorator
================================================
FILE: gplay_scraper/utils/helpers.py
================================================
"""Helper functions for data processing and manipulation.
This module contains utility functions for:
- Text unescaping and cleaning
- JSON string cleaning
- Date parsing and calculations
- Install metrics calculations
"""
import re
import json
import os
from html import unescape
from typing import Any, List, Optional, Dict
from datetime import datetime, timezone
from urllib.parse import urlparse
from .constants import PHONE_PREFIXES
def unescape_text(s: Optional[str]) -> Optional[str]:
"""Unescape HTML entities and remove HTML tags from text.
Args:
s: Input string with HTML
Returns:
Cleaned text without HTML tags
"""
if s is None:
return None
text = s.replace("
", "\n").replace("
", "\n").replace("
", "\n")
text = text.replace("", "").replace("", "")
text = text.replace("", "").replace("", "")
text = text.replace("", "").replace("", "")
text = text.replace("", "").replace("", "")
text = text.replace("", "").replace("", "")
text = re.sub(r'<[^>]+>', '', text)
return unescape(text).strip()
def clean_json_string(json_str: str) -> str:
"""Clean malformed JSON string from Google Play Store.
Args:
json_str: Raw JSON string
Returns:
Cleaned JSON string
"""
json_str = re.sub(r',\s*sideChannel:\s*\{\}', '', json_str)
json_str = re.sub(r'([{,]\s*)([a-zA-Z_$][a-zA-Z0-9_$]*)\s*:', r'\1"\2":', json_str)
json_str = re.sub(r'\bfunction\s*\([^)]*\)\s*\{[^}]*\}', 'null', json_str)
json_str = re.sub(r'\bundefined\b', 'null', json_str)
json_str = re.sub(r":\s*'([^']*)'", r': "\1"', json_str)
json_str = re.sub(r'(\])\s*(\[)', r'\1,\2', json_str)
json_str = re.sub(r'(\})\s*(\{)', r'\1,\2', json_str)
json_str = re.sub(r',(\s*[}\]])', r'\1', json_str)
json_str = re.sub(r',,+', ',', json_str)
json_str = re.sub(r':\s*\$([0-9.]+)', r': "$\1"', json_str)
json_str = re.sub(r'"version"\s*:\s*([0-9.]+)(?=\s*[,}])', r'"version": "\1"', json_str)
return json_str
def alternative_json_clean(json_str: str) -> str:
"""Alternative JSON cleaning method using bracket matching.
Fallback method for cleaning malformed JSON when the primary cleaning fails.
Uses bracket counting to extract valid JSON arrays from complex structures.
Args:
json_str: Raw JSON string from Google Play Store
Returns:
Cleaned JSON string ready for parsing
Process:
1. Find 'data:' marker in the string
2. Use bracket counting to extract complete array
3. Wrap in standard ds:5 format
4. Apply basic cleaning as fallback
Example:
>>> alternative_json_clean('data: [1,2,3] extra content')
'{"key": "ds:5", "hash": "13", "data": [1,2,3]}'
"""
# Look for data array marker
data_start = json_str.find('data:')
if data_start != -1:
bracket_start = json_str.find('[', data_start)
if bracket_start != -1:
bracket_count = 0
pos = bracket_start
# Count brackets to find complete array
while pos < len(json_str):
if json_str[pos] == '[':
bracket_count += 1
elif json_str[pos] == ']':
bracket_count -= 1
if bracket_count == 0:
data_end = pos + 1
break
pos += 1
# Extract and parse the complete array
if bracket_count == 0:
data_array = json_str[bracket_start:data_end]
try:
parsed_array = json.loads(data_array)
# Wrap in standard ds:5 format
return json.dumps({
"key": "ds:5",
"hash": "13",
"data": parsed_array
})
except json.JSONDecodeError:
pass
# Fallback: basic cleaning
json_str = re.sub(r'\bNaN\b', 'null', json_str)
return clean_json_string(json_str)
def parse_release_date(release_date_str: Optional[str]) -> Optional[datetime]:
"""Parse release date string to datetime object.
Converts Google Play Store date format to Python datetime object
for date calculations and comparisons.
Args:
release_date_str: Date string in format 'Mon DD, YYYY' (e.g., 'Jan 15, 2020')
Returns:
Datetime object or None if parsing fails
Example:
>>> parse_release_date('Jan 15, 2020')
datetime.datetime(2020, 1, 15, 0, 0)
>>> parse_release_date('invalid date')
None
"""
if release_date_str is None:
return None
try:
# Parse Google Play Store date format
return datetime.strptime(release_date_str, "%b %d, %Y")
except (ValueError, TypeError):
# Return None for invalid date formats
return None
def calculate_app_age(release_date_str: Optional[str], current_date: datetime) -> Optional[int]:
"""Calculate app age in days since release.
Computes the number of days between app release date and current date,
useful for analyzing app maturity and growth metrics.
Args:
release_date_str: Release date string (e.g., 'Jan 15, 2020')
current_date: Current date for calculation (usually datetime.now())
Returns:
Number of days since release (non-negative) or None if date invalid
Example:
>>> from datetime import datetime
>>> current = datetime(2023, 1, 15)
>>> calculate_app_age('Jan 15, 2020', current)
1095 # 3 years = ~1095 days
"""
release_date = parse_release_date(release_date_str)
if release_date is None:
return None
# Handle timezone differences
if current_date.tzinfo is not None and release_date.tzinfo is None:
release_date = release_date.replace(tzinfo=timezone.utc)
# Calculate days difference
days_since_release = (current_date - release_date).days
# Ensure non-negative result
return max(0, days_since_release)
def parse_installs_string(installs_str: str) -> Optional[int]:
"""Parse install count string to integer.
Converts Google Play Store install count strings (with commas and plus signs)
to numeric values for calculations and comparisons.
Args:
installs_str: Install count string (e.g., '1,000,000+', '500,000')
Returns:
Integer install count or None if parsing fails
Example:
>>> parse_installs_string('1,000,000+')
1000000
>>> parse_installs_string('500,000')
500000
>>> parse_installs_string('invalid')
None
"""
if installs_str is None:
return None
# Remove formatting characters
cleaned_str = installs_str.replace(',', '').replace('+', '')
try:
return int(cleaned_str)
except (ValueError, TypeError):
return None
def calculate_daily_installs(install_count, release_date_str: Optional[str], current_date: datetime) -> Optional[int]:
"""Calculate average daily installs since release.
Computes the average number of installs per day since app release,
providing insight into app growth rate and popularity trends.
Args:
install_count: Total install count (int or string like '1,000,000+')
release_date_str: Release date string (e.g., 'Jan 15, 2020')
current_date: Current date for calculation
Returns:
Average daily installs (integer) or None if calculation impossible
Example:
>>> from datetime import datetime
>>> current = datetime(2023, 1, 15)
>>> calculate_daily_installs(1000000, 'Jan 15, 2020', current)
913 # ~1M installs over ~1095 days
"""
# Convert string install count to integer if needed
if isinstance(install_count, str):
install_count = parse_installs_string(install_count)
if install_count is None or release_date_str is None:
return None
release_date = parse_release_date(release_date_str)
if release_date is None:
return None
# Handle timezone differences
if current_date.tzinfo is not None and release_date.tzinfo is None:
release_date = release_date.replace(tzinfo=timezone.utc)
days_since_release = (current_date - release_date).days
if days_since_release <= 0:
return 0
# Calculate average daily installs
return int(install_count / days_since_release)
def calculate_monthly_installs(install_count, release_date_str: Optional[str], current_date: datetime) -> Optional[int]:
"""Calculate average monthly installs since release.
Computes the average number of installs per month since app release,
useful for understanding monthly growth patterns and trends.
Args:
install_count: Total install count (int or string like '1,000,000+')
release_date_str: Release date string (e.g., 'Jan 15, 2020')
current_date: Current date for calculation
Returns:
Average monthly installs (integer) or None if calculation impossible
Note:
Uses 30.44 days per month (365.25/12) for accurate monthly calculations
Example:
>>> from datetime import datetime
>>> current = datetime(2023, 1, 15)
>>> calculate_monthly_installs(1000000, 'Jan 15, 2020', current)
27777 # ~1M installs over ~36 months
"""
# Convert string install count to integer if needed
if isinstance(install_count, str):
install_count = parse_installs_string(install_count)
if install_count is None or release_date_str is None:
return None
release_date = parse_release_date(release_date_str)
if release_date is None:
return None
# Handle timezone differences
if current_date.tzinfo is not None and release_date.tzinfo is None:
release_date = release_date.replace(tzinfo=timezone.utc)
days_since_release = (current_date - release_date).days
if days_since_release <= 0:
return 0
# Convert days to months (using average month length)
months_since_release = days_since_release / 30.44 # 365.25/12
return int(install_count / months_since_release)
def tamp_to_date(value) -> str:
"""Convert timestamp to date format 'Jul 21, 2023' if value is a timestamp.
Detects Unix timestamps and converts them to human-readable date format.
Non-timestamp values are returned unchanged.
Args:
value: Value to check and convert (int, float, or any other type)
Returns:
Formatted date string (e.g., 'Jul 21, 2023') if timestamp, otherwise original value
Example:
>>> tamp_to_date(1642780800)
'Jan 21, 2022'
>>> tamp_to_date('not a timestamp')
'not a timestamp'
"""
# Check if value looks like a Unix timestamp (> 1 billion = after 2001)
if isinstance(value, (int, float)) and value > 1000000000:
try:
dt = datetime.fromtimestamp(value)
return dt.strftime("%b %d, %Y")
except (ValueError, OSError):
# Invalid timestamp, return original value
pass
return value
def get_publisher_country(phone: Optional[str], address: Optional[str]) -> str:
"""Determine publisher country from phone and address information.
Analyzes developer contact information to determine their likely country
of origin by extracting country codes from phone numbers and addresses.
Args:
phone: Developer phone number (e.g., '+1-555-123-4567')
address: Developer address string (may contain country name)
Returns:
Country name(s) or 'Unknown' if cannot be determined
Examples:
>>> get_publisher_country('+1-555-123-4567', None)
'United States'
>>> get_publisher_country(None, 'London, UK')
'United Kingdom'
>>> get_publisher_country('+1-555-123', 'Berlin, Germany')
'United States/Germany'
"""
# Extract country codes from phone and address
phone_code = pho_count(phone) if phone else None
address_code = add_count(address) if address else None
# Convert country codes to readable names
code_to_name = {item[1]: item[2].title() for item in PHONE_PREFIXES}
phone_country = code_to_name.get(phone_code) if phone_code else None
address_country = code_to_name.get(address_code) if address_code else None
# Determine final country based on available information
if not phone_country and not address_country:
return "Unknown"
elif phone_country and not address_country:
return phone_country
elif address_country and not phone_country:
return address_country
elif phone_country == address_country:
return phone_country
else:
# Different countries detected, show both
return f"{phone_country}/{address_country}"
def add_count(address):
"""Extract country code from address string.
Parses address text to find country names or codes and returns
the corresponding country code for country identification.
Args:
address: Address string with country information (e.g., 'London, UK')
Returns:
Country code (e.g., 'gb') or None if not found
Example:
>>> add_count('123 Main St\nLondon, UK')
'gb'
>>> add_count('Berlin, Germany')
'de'
>>> add_count('Unknown location')
None
"""
if not address:
return None
# Split address into parts (lines)
parts = address.split('\n')
# Create lookup dictionaries from phone prefixes data
code_to_code = {item[1].lower(): item[1] for item in PHONE_PREFIXES}
name_to_code = {item[2].lower(): item[1] for item in PHONE_PREFIXES}
# Check each part of the address for country matches
for part in parts:
part_lower = part.strip().lower()
# Check for exact country code match
if part_lower in code_to_code:
return code_to_code[part_lower]
# Check for country name match
if part_lower in name_to_code:
return name_to_code[part_lower]
return None
def pho_count(phone):
"""Extract country code from phone number.
Analyzes phone number format to determine the country code by matching
against known international phone prefixes.
Args:
phone: Phone number string (e.g., '+1-555-123-4567', '44-20-1234-5678')
Returns:
Country code (e.g., 'us', 'gb') or None if not found
Example:
>>> pho_count('+1-555-123-4567')
'us'
>>> pho_count('44-20-1234-5678')
'gb'
>>> pho_count('invalid')
None
"""
if not isinstance(phone, str) or len(phone) < 10:
return None
# Remove leading '+' if present
phone = phone.lstrip('+')
# Create lookup dictionary from phone prefixes
prefix_to_code = {item[0]: item[1] for item in PHONE_PREFIXES}
# Handle North American numbers (starting with 1)
if phone.startswith('1'):
try:
# North American numbers use 4-digit area codes
prefix = int(phone[:4])
return prefix_to_code.get(prefix)
except ValueError:
return None
else:
# Try different prefix lengths (longest first)
for length in range(7, 0, -1):
try:
prefix = int(phone[:length])
if prefix in prefix_to_code:
return prefix_to_code[prefix]
except ValueError:
continue
return None
================================================
FILE: gplay_scraper/utils/http_client.py
================================================
"""HTTP client with support for 7 different libraries and automatic fallback.
This module provides a unified HTTP client interface that supports:
- requests
- curl_cffi
- tls_client
- httpx
- urllib3
- cloudscraper
- aiohttp
With automatic fallback if the primary client fails.
"""
import time
import logging
from typing import Optional
from urllib.parse import quote
from ..config import Config
from ..exceptions import AppNotFoundError, NetworkError
logger = logging.getLogger(__name__)
class HttpClient:
"""HTTP client with automatic fallback support for 7 libraries.
Provides a unified interface for making HTTP requests using various client libraries.
Automatically falls back to alternative clients if the preferred one fails or is unavailable.
Supported Clients:
- requests: Most compatible, widely used
- curl_cffi: Best for bypassing anti-bot protections
- tls_client: Advanced TLS fingerprinting
- urllib3: Low-level HTTP with connection pooling
- cloudscraper: Automatic Cloudflare bypass
- aiohttp: Asynchronous HTTP operations
- httpx: Modern HTTP client with HTTP/2 support
Features:
- Automatic client fallback on failures
- Rate limiting to prevent blocks
- Browser impersonation capabilities
- Connection pooling and reuse
- Comprehensive error handling
"""
def __init__(self, rate_limit_delay: float = None, client_type: str = None):
"""Initialize HTTP client with specified or default client type.
Args:
rate_limit_delay: Delay between requests in seconds (default: 1.0)
client_type: HTTP client to use - options:
'requests', 'curl_cffi', 'tls_client', 'urllib3',
'cloudscraper', 'aiohttp', 'httpx' (default: 'requests')
"""
self.headers = Config.get_headers()
self.timeout = Config.DEFAULT_TIMEOUT
self.rate_limit_delay = rate_limit_delay or Config.RATE_LIMIT_DELAY
self.last_request_time = 0
self.client_type = client_type or Config.DEFAULT_HTTP_CLIENT
self.available_clients = ["requests", "curl_cffi", "tls_client", "urllib3", "cloudscraper", "aiohttp", "httpx"]
self.current_client_index = 0
self._setup_client()
def _setup_client(self):
"""Setup HTTP client based on client_type with automatic fallback.
Initializes the specified HTTP client library. If the requested client
is not available, automatically falls back to the next available client
in the priority order.
Priority Order:
1. requests (default, most compatible)
2. curl_cffi (best for bypassing blocks)
3. tls_client (advanced TLS fingerprinting)
4. urllib3 (low-level HTTP)
5. cloudscraper (anti-bot protection)
6. aiohttp (async support)
7. httpx (modern HTTP client)
"""
# Try to initialize the specified client, fallback to next if unavailable
if self.client_type == "requests" or self.client_type is None:
self._try_requests()
elif self.client_type == "curl_cffi":
self._try_curl_cffi()
elif self.client_type == "tls_client":
self._try_tls_client()
elif self.client_type == "urllib3":
self._try_urllib3()
elif self.client_type == "cloudscraper":
self._try_cloudscraper()
elif self.client_type == "aiohttp":
self._try_aiohttp()
elif self.client_type == "httpx":
self._try_httpx()
else:
# Default fallback to requests
self._try_requests()
def _try_requests(self):
"""Try to initialize requests client, fallback to curl_cffi if unavailable.
Requests is the most widely used HTTP library for Python, providing
simple and reliable HTTP functionality. Used as the default client.
Fallback: curl_cffi (if requests unavailable)
"""
try:
import requests
self.client = requests
self.client_type = "requests"
except ImportError:
# Fallback to next available client
self._try_curl_cffi()
def _try_curl_cffi(self):
"""Try to initialize curl_cffi client, fallback to tls_client if unavailable.
curl_cffi provides libcurl bindings with browser impersonation capabilities,
making it excellent for bypassing anti-bot protections by mimicking real browsers.
Features:
- Browser impersonation (Chrome 110)
- Advanced TLS fingerprinting
- Better success rate against blocks
Fallback: tls_client (if curl_cffi unavailable)
"""
try:
from curl_cffi import requests as curl_requests
# Impersonate Chrome 110 for better compatibility
self.client = curl_requests.Session(impersonate="chrome110")
self.client_type = "curl_cffi"
except ImportError:
# Fallback to next available client
self._try_tls_client()
def _try_tls_client(self):
"""Try to initialize tls_client, fallback to urllib3 if unavailable.
tls_client provides advanced TLS fingerprinting and browser simulation
capabilities, useful for bypassing sophisticated detection systems.
Features:
- Chrome 112 client identifier
- Random TLS extension ordering
- Advanced fingerprint randomization
Fallback: urllib3 (if tls_client unavailable)
"""
try:
import tls_client
self.client = tls_client.Session(
client_identifier="chrome112", # Simulate Chrome 112
random_tls_extension_order=True # Randomize TLS fingerprint
)
self.client_type = "tls_client"
except ImportError:
# Fallback to next available client
self._try_urllib3()
def _try_urllib3(self):
"""Try to initialize urllib3 client, fallback to cloudscraper if unavailable.
urllib3 is a powerful HTTP client library that provides connection pooling,
thread safety, and many other features. It's the foundation for requests.
Features:
- Connection pooling for better performance
- Thread-safe operations
- Low-level HTTP control
Fallback: cloudscraper (if urllib3 unavailable)
"""
try:
import urllib3
# Use PoolManager for connection pooling
self.client = urllib3.PoolManager()
self.client_type = "urllib3"
except ImportError:
# Fallback to next available client
self._try_cloudscraper()
def _try_cloudscraper(self):
"""Try to initialize cloudscraper client, fallback to aiohttp if unavailable.
cloudscraper is designed to bypass Cloudflare's anti-bot protection
and other similar security measures automatically.
Features:
- Automatic Cloudflare bypass
- JavaScript challenge solving
- Anti-bot protection circumvention
Fallback: aiohttp (if cloudscraper unavailable)
"""
try:
import cloudscraper
# Create scraper with automatic anti-bot bypass
self.client = cloudscraper.create_scraper()
self.client_type = "cloudscraper"
except ImportError:
# Fallback to next available client
self._try_aiohttp()
def _try_aiohttp(self):
"""Try to initialize aiohttp client, fallback to httpx if unavailable.
aiohttp provides asynchronous HTTP client/server functionality,
allowing for better performance with concurrent requests.
Features:
- Asynchronous operations
- Better performance for multiple requests
- WebSocket support
Note: Used synchronously via asyncio.run() in this implementation
Fallback: httpx (if aiohttp unavailable)
"""
try:
import aiohttp
# Store aiohttp module for async operations
self.client = aiohttp
self.client_type = "aiohttp"
except ImportError:
# Fallback to next available client
self._try_httpx()
def _try_httpx(self):
"""Try to initialize httpx client, raise error if unavailable.
httpx is a modern HTTP client library with async support and HTTP/2 capabilities.
It provides a requests-compatible API with additional features.
Features:
- HTTP/2 support
- Async and sync APIs
- Modern Python features
- Requests-compatible interface
Raises:
ImportError: If no HTTP client libraries are available
"""
try:
import httpx
# Create client with configured timeout
self.client = httpx.Client(timeout=self.timeout)
self.client_type = "httpx"
except ImportError:
# No more fallback options available
raise ImportError(Config.ERROR_MESSAGES["NO_HTTP_CLIENT"])
def fetch_app_page(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch app details page from Google Play Store.
Retrieves the HTML content of an app's details page, which contains
all the app information including ratings, reviews, description, etc.
Args:
app_id: Google Play app ID (e.g., 'com.whatsapp')
lang: Language code for localization (e.g., 'en', 'es')
country: Country code for regional content (e.g., 'us', 'uk')
Returns:
HTML content of app page containing embedded JSON data
Raises:
AppNotFoundError: If app not found (404 error)
NetworkError: If request fails due to network issues
Example:
html = client.fetch_app_page('com.whatsapp', 'en', 'us')
"""
self.rate_limit()
url = f"{Config.PLAY_STORE_BASE_URL}{Config.APP_DETAILS_ENDPOINT}?id={app_id}&hl={lang}&gl={country}"
try:
response = self._make_request("GET", url)
return response.text
except Exception as e:
if self._is_404_error(e):
raise AppNotFoundError(Config.ERROR_MESSAGES["APP_NOT_FOUND"].format(app_id=app_id))
# Retry without country parameter
url = f"{Config.PLAY_STORE_BASE_URL}{Config.APP_DETAILS_ENDPOINT}?id={app_id}&hl={lang}"
try:
response = self._make_request("GET", url)
return response.text
except Exception as e2:
if self._is_404_error(e2):
raise AppNotFoundError(Config.ERROR_MESSAGES["APP_NOT_FOUND"].format(app_id=app_id))
logger.error(Config.ERROR_MESSAGES["APP_FETCH_FAILED"].format(app_id=app_id, error=e))
raise NetworkError(Config.ERROR_MESSAGES["APP_FETCH_FAILED"].format(app_id=app_id, error=e))
def fetch_app_page_no_locale(self, app_id: str) -> str:
"""Fetch app page without hl/gl parameters for fallback data.
Args:
app_id: Google Play app ID
Returns:
HTML content of app page
"""
self.rate_limit()
url = f"{Config.PLAY_STORE_BASE_URL}{Config.APP_DETAILS_ENDPOINT}?id={app_id}"
try:
response = self._make_request("GET", url)
return response.text
except Exception as e:
logger.error(f"Fallback fetch failed for {app_id}: {e}")
return ""
def fetch_search_page(self, query: str = None, token: str = None, needed: int = None, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch search results from Google Play Store (initial or paginated).
Args:
query: Search query string (for initial search)
token: Pagination token (for paginated search)
needed: Number of results needed (for pagination)
lang: Language code
country: Country code
Returns:
HTML content (initial) or raw API response (pagination)
Raises:
AppNotFoundError: If search fails
NetworkError: If request fails
"""
self.rate_limit()
# Pagination request
if token and needed:
url = f"{Config.PLAY_STORE_BASE_URL}/_/PlayStoreUi/data/batchexecute"
params = f"rpcids=qnKhOb&source-path=%2Fwork%2Fsearch&hl={lang}&gl={country}"
body = f'f.req=%5B%5B%5B%22qnKhOb%22%2C%22%5B%5Bnull%2C%5B%5B10%2C%5B10%2C{needed}%5D%5D%2Ctrue%2Cnull%2C%5B96%2C27%2C4%2C8%2C57%2C30%2C110%2C79%2C11%2C16%2C49%2C1%2C3%2C9%2C12%2C104%2C55%2C56%2C51%2C10%2C34%2C77%5D%5D%2Cnull%2C%5C%22{token}%5C%22%5D%5D%22%2Cnull%2C%22generic%22%5D%5D%5D'
headers = {
**self.headers,
"Content-Type": "application/x-www-form-urlencoded;charset=UTF-8"
}
try:
response = self._make_request("POST", f"{url}?{params}", data=body, headers=headers)
return response.text
except Exception as e:
logger.error(Config.ERROR_MESSAGES["SEARCH_PAGINATION_FAILED"].format(error=e))
raise NetworkError(Config.ERROR_MESSAGES["SEARCH_PAGINATION_FAILED"].format(error=e))
# Initial search request
elif query:
encoded_query = quote(query)
url = f"{Config.PLAY_STORE_BASE_URL}/work/search?q={encoded_query}&hl={lang}&gl={country}&price=0"
try:
response = self._make_request("GET", url)
return response.text
except Exception as e:
if self._is_404_error(e):
raise AppNotFoundError(Config.ERROR_MESSAGES["SEARCH_NOT_FOUND"].format(query=query))
url = f"{Config.PLAY_STORE_BASE_URL}/work/search?q={encoded_query}&hl={lang}&price=0"
try:
response = self._make_request("GET", url)
return response.text
except Exception as e2:
if self._is_404_error(e2):
raise AppNotFoundError(Config.ERROR_MESSAGES["SEARCH_NOT_FOUND"].format(query=query))
logger.error(Config.ERROR_MESSAGES["SEARCH_FETCH_FAILED"].format(query=query, error=e))
raise NetworkError(Config.ERROR_MESSAGES["SEARCH_FETCH_FAILED"].format(query=query, error=e))
else:
raise ValueError("Either query or (token and needed) must be provided")
def fetch_reviews_batch(self, app_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY,
sort: int = Config.DEFAULT_REVIEWS_SORT, batch_count: int = Config.DEFAULT_REVIEWS_BATCH_SIZE, token: str = None) -> str:
"""Fetch single batch of reviews from Google Play Store API.
Args:
app_id: Google Play app ID
lang: Language code
country: Country code
sort: Sort order (1=RELEVANT, 2=NEWEST, 3=RATING)
batch_count: Number of reviews per batch
token: Pagination token for next batch
Returns:
Raw API response text
Raises:
AppNotFoundError: If reviews not found
NetworkError: If request fails
"""
self.rate_limit()
url = f"{Config.PLAY_STORE_BASE_URL}{Config.BATCHEXECUTE_ENDPOINT}?hl={lang}&gl={country}"
headers = {
**self.headers,
"content-type": "application/x-www-form-urlencoded"
}
if token:
payload = f"f.req=%5B%5B%5B%22oCPfdb%22%2C%22%5Bnull%2C%5B2%2C{sort}%2C%5B{batch_count}%2Cnull%2C%5C%22{token}%5C%22%5D%2Cnull%2C%5Bnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%5D%5D%2C%5B%5C%22{app_id}%5C%22%2C7%5D%5D%22%2Cnull%2C%22generic%22%5D%5D%5D"
else:
payload = f"f.req=%5B%5B%5B%22oCPfdb%22%2C%22%5Bnull%2C%5B2%2C{sort}%2C%5B{batch_count}%5D%2Cnull%2C%5Bnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%5D%5D%2C%5B%5C%22{app_id}%5C%22%2C7%5D%5D%22%2Cnull%2C%22generic%22%5D%5D%5D"
try:
response = self._make_request("POST", url, data=payload, headers=headers)
return response.text
except Exception as e:
if self._is_404_error(e):
raise AppNotFoundError(Config.ERROR_MESSAGES["REVIEWS_NOT_FOUND"].format(app_id=app_id))
logger.error(Config.ERROR_MESSAGES["REVIEWS_FETCH_FAILED"].format(app_id=app_id, error=e))
raise NetworkError(Config.ERROR_MESSAGES["REVIEWS_FETCH_FAILED"].format(app_id=app_id, error=e))
def fetch_developer_page(self, dev_id: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch developer portfolio page from Google Play Store.
Args:
dev_id: Developer ID (numeric or string)
lang: Language code
country: Country code
Returns:
HTML content of developer page
Raises:
AppNotFoundError: If developer not found
NetworkError: If request fails
"""
self.rate_limit()
if dev_id.isdigit():
url = f"{Config.PLAY_STORE_BASE_URL}{Config.DEVELOPER_NUMERIC_ENDPOINT}?id={quote(dev_id)}&hl={lang}&gl={country}"
else:
url = f"{Config.PLAY_STORE_BASE_URL}{Config.DEVELOPER_STRING_ENDPOINT}?id={quote(dev_id)}&hl={lang}&gl={country}"
try:
response = self._make_request("GET", url)
return response.text
except Exception as e:
if self._is_404_error(e):
raise AppNotFoundError(Config.ERROR_MESSAGES["DEVELOPER_NOT_FOUND"].format(dev_id=dev_id))
if dev_id.isdigit():
url = f"{Config.PLAY_STORE_BASE_URL}{Config.DEVELOPER_NUMERIC_ENDPOINT}?id={quote(dev_id)}&hl={lang}"
else:
url = f"{Config.PLAY_STORE_BASE_URL}{Config.DEVELOPER_STRING_ENDPOINT}?id={quote(dev_id)}&hl={lang}"
try:
response = self._make_request("GET", url)
return response.text
except Exception as e2:
if self._is_404_error(e2):
raise AppNotFoundError(Config.ERROR_MESSAGES["DEVELOPER_NOT_FOUND"].format(dev_id=dev_id))
logger.error(Config.ERROR_MESSAGES["DEVELOPER_FETCH_FAILED"].format(dev_id=dev_id, error=e))
raise NetworkError(Config.ERROR_MESSAGES["DEVELOPER_FETCH_FAILED"].format(dev_id=dev_id, error=e))
def fetch_cluster_page(self, cluster_url: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch cluster page (similar apps collection) from Google Play Store.
Args:
cluster_url: Cluster URL path
lang: Language code
country: Country code
Returns:
HTML content of cluster page
Raises:
AppNotFoundError: If cluster not found
NetworkError: If request fails
"""
self.rate_limit()
url = f"{Config.PLAY_STORE_BASE_URL}{cluster_url}&gl={country}&hl={lang}"
try:
response = self._make_request("GET", url)
return response.text
except Exception as e:
if self._is_404_error(e):
raise AppNotFoundError(Config.ERROR_MESSAGES["CLUSTER_NOT_FOUND"].format(cluster_url=cluster_url))
logger.error(Config.ERROR_MESSAGES["CLUSTER_FETCH_FAILED"].format(error=e))
raise NetworkError(Config.ERROR_MESSAGES["CLUSTER_FETCH_FAILED"].format(error=e))
def fetch_list_page(self, collection: str, category: str = Config.DEFAULT_LIST_CATEGORY, count: int = Config.DEFAULT_LIST_COUNT, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch top charts list page from Google Play Store.
Args:
collection: Collection type (topselling_free, topselling_paid, topgrossing)
category: App category
count: Number of apps to fetch
lang: Language code
country: Country code
Returns:
Raw API response text
Raises:
AppNotFoundError: If list not found
NetworkError: If request fails
"""
self.rate_limit()
body = f'f.req=%5B%5B%5B%22vyAe2%22%2C%22%5B%5Bnull%2C%5B%5B8%2C%5B20%2C{count}%5D%5D%2Ctrue%2Cnull%2C%5B64%2C1%2C195%2C71%2C8%2C72%2C9%2C10%2C11%2C139%2C12%2C16%2C145%2C148%2C150%2C151%2C152%2C27%2C30%2C31%2C96%2C32%2C34%2C163%2C100%2C165%2C104%2C169%2C108%2C110%2C113%2C55%2C56%2C57%2C122%5D%2C%5Bnull%2Cnull%2C%5B%5B%5Btrue%5D%2Cnull%2C%5B%5Bnull%2C%5B%5D%5D%5D%2Cnull%2Cnull%2Cnull%2Cnull%2C%5Bnull%2C2%5D%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2C%5B1%5D%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2C%5B1%5D%5D%2C%5Bnull%2C%5B%5Bnull%2C%5B%5D%5D%5D%5D%2C%5Bnull%2C%5B%5Bnull%2C%5B%5D%5D%5D%2Cnull%2C%5Btrue%5D%5D%2C%5Bnull%2C%5B%5Bnull%2C%5B%5D%5D%5D%5D%2Cnull%2Cnull%2Cnull%2Cnull%2C%5B%5B%5Bnull%2C%5B%5D%5D%5D%5D%2C%5B%5B%5Bnull%2C%5B%5D%5D%5D%5D%5D%2C%5B%5B%5B%5B7%2C1%5D%2C%5B%5B1%2C73%2C96%2C103%2C97%2C58%2C50%2C92%2C52%2C112%2C69%2C19%2C31%2C101%2C123%2C74%2C49%2C80%2C38%2C20%2C10%2C14%2C79%2C43%2C42%2C139%5D%5D%5D%5D%5D%5D%2Cnull%2Cnull%2C%5B%5B%5B1%2C2%5D%2C%5B10%2C8%2C9%5D%2C%5B%5D%2C%5B%5D%5D%5D%5D%2C%5B2%2C%5C%22{collection}%5C%22%2C%5C%22{category}%5C%22%5D%5D%5D%22%2Cnull%2C%22generic%22%5D%5D%5D&at=AFSRYlx8XZfN8-O-IKASbNBDkB6T%3A1655531200971&'
url = f"{Config.PLAY_STORE_BASE_URL}{Config.BATCHEXECUTE_ENDPOINT}?rpcids=vyAe2&source-path=%2Fstore%2Fapps&hl={lang}&gl={country}"
headers = {
**self.headers,
"Content-Type": "application/x-www-form-urlencoded;charset=UTF-8"
}
try:
response = self._make_request("POST", url, data=body, headers=headers)
return response.text
except Exception as e:
if self._is_404_error(e):
raise AppNotFoundError(Config.ERROR_MESSAGES["LIST_NOT_FOUND"].format(collection=collection, category=category))
logger.error(Config.ERROR_MESSAGES["LIST_FETCH_FAILED"].format(error=e))
raise NetworkError(Config.ERROR_MESSAGES["LIST_FETCH_FAILED"].format(error=e))
def fetch_suggest_page(self, term: str, lang: str = Config.DEFAULT_LANGUAGE, country: str = Config.DEFAULT_COUNTRY) -> str:
"""Fetch search suggestions from Google Play Store.
Args:
term: Search term for suggestions
lang: Language code
country: Country code
Returns:
Raw API response text
Raises:
AppNotFoundError: If suggestions not found
NetworkError: If request fails
"""
self.rate_limit()
encoded_term = quote(term)
url = f"{Config.PLAY_STORE_BASE_URL}{Config.BATCHEXECUTE_ENDPOINT}?rpcids=IJ4APc&f.sid=-697906427155521722&bl=boq_playuiserver_20190903.08_p0&hl={lang}&gl={country}&authuser&soc-app=121&soc-platform=1&soc-device=1&_reqid=1065213"
body = f"f.req=%5B%5B%5B%22IJ4APc%22%2C%22%5B%5Bnull%2C%5B%5C%22{encoded_term}%5C%22%5D%2C%5B10%5D%2C%5B2%5D%2C4%5D%5D%22%5D%5D%5D"
headers = {
**self.headers,
"Content-Type": "application/x-www-form-urlencoded;charset=UTF-8"
}
try:
response = self._make_request("POST", url, data=body, headers=headers)
return response.text
except Exception as e:
if self._is_404_error(e):
raise AppNotFoundError(Config.ERROR_MESSAGES["SUGGEST_NOT_FOUND"].format(term=term))
logger.error(Config.ERROR_MESSAGES["SUGGEST_FETCH_FAILED"].format(term=term, error=e))
raise NetworkError(Config.ERROR_MESSAGES["SUGGEST_FETCH_FAILED"].format(term=term, error=e))
def _make_request(self, method: str, url: str, **kwargs):
"""Make HTTP request with automatic client fallback.
Attempts to make an HTTP request using the configured client. If the request
fails, automatically tries other available HTTP clients in priority order
until one succeeds or all clients are exhausted.
Args:
method: HTTP method (GET or POST)
url: Request URL
**kwargs: Additional request parameters (data, headers, etc.)
Returns:
Response object with .text attribute
Raises:
Exception: If all HTTP clients fail to make the request
Example:
response = self._make_request("GET", "https://example.com")
content = response.text
"""
# Build list of clients to try, starting with preferred client
clients_to_try = [self.client_type]
all_clients = ["requests", "curl_cffi", "tls_client", "urllib3", "cloudscraper", "aiohttp", "httpx"]
# Add remaining clients as fallback options
for client in all_clients:
if client != self.client_type:
clients_to_try.append(client)
last_error = None
# Try each client until one succeeds
for client_type in clients_to_try:
try:
return self._try_request_with_client(client_type, method, url, **kwargs)
except Exception as e:
last_error = e
logger.warning(Config.ERROR_MESSAGES["CLIENT_FAILED_TRYING_NEXT"].format(client_type=client_type, error=e))
continue
# All clients failed, raise the last error
raise last_error
def _try_request_with_client(self, client_type: str, method: str, url: str, **kwargs):
"""Attempt request with specific HTTP client.
Tries to make an HTTP request using the specified client library.
Each client has its own implementation details and capabilities.
Args:
client_type: HTTP client name (requests, curl_cffi, etc.)
method: HTTP method (GET or POST)
url: Request URL
**kwargs: Additional request parameters (data, headers, timeout)
Returns:
Response object with .text attribute and .status_code
Raises:
Exception: If client unavailable or request fails
Note:
Different clients may have different response object structures,
so this method normalizes them to a common interface.
"""
headers = kwargs.get('headers', self.headers)
if client_type == "requests":
try:
import requests
if method == "GET":
response = requests.get(url, headers=headers, timeout=self.timeout)
else:
response = requests.post(url, data=kwargs.get('data'), headers=headers, timeout=self.timeout)
response.raise_for_status()
return response
except ImportError:
raise Exception(Config.ERROR_MESSAGES["HTTP_CLIENT_NOT_AVAILABLE"].format(client="requests"))
elif client_type == "curl_cffi":
try:
from curl_cffi import requests as curl_requests
session = curl_requests.Session(impersonate="chrome110")
if method == "GET":
response = session.get(url, headers=headers, timeout=self.timeout)
else:
response = session.post(url, data=kwargs.get('data'), headers=headers, timeout=self.timeout)
response.raise_for_status()
return response
except ImportError:
raise Exception(Config.ERROR_MESSAGES["HTTP_CLIENT_NOT_AVAILABLE"].format(client="curl_cffi"))
elif client_type == "tls_client":
try:
import tls_client
session = tls_client.Session(client_identifier="chrome112", random_tls_extension_order=True)
if method == "GET":
response = session.get(url, headers=headers)
else:
response = session.post(url, data=kwargs.get('data'), headers=headers)
if response.status_code >= 400:
raise Exception(Config.ERROR_MESSAGES["HTTP_ERROR"].format(status_code=response.status_code))
return response
except ImportError:
raise Exception(Config.ERROR_MESSAGES["HTTP_CLIENT_NOT_AVAILABLE"].format(client="tls_client"))
elif client_type == "httpx":
try:
import httpx
with httpx.Client(timeout=self.timeout) as client:
if method == "GET":
response = client.get(url, headers=headers)
else:
response = client.post(url, data=kwargs.get('data'), headers=headers)
response.raise_for_status()
return response
except ImportError:
raise Exception(Config.ERROR_MESSAGES["HTTP_CLIENT_NOT_AVAILABLE"].format(client="httpx"))
elif client_type == "urllib3":
try:
import urllib3
http = urllib3.PoolManager()
if method == "GET":
response = http.request('GET', url, headers=headers)
else:
response = http.request('POST', url, body=kwargs.get('data'), headers=headers)
if response.status >= 400:
raise Exception(Config.ERROR_MESSAGES["HTTP_ERROR"].format(status_code=response.status))
class MockResponse:
def __init__(self, data, status):
self.text = data.decode('utf-8')
self.status_code = status
def raise_for_status(self):
pass
return MockResponse(response.data, response.status)
except ImportError:
raise Exception(Config.ERROR_MESSAGES["HTTP_CLIENT_NOT_AVAILABLE"].format(client="urllib3"))
elif client_type == "cloudscraper":
try:
import cloudscraper
scraper = cloudscraper.create_scraper()
if method == "GET":
response = scraper.get(url, headers=headers, timeout=self.timeout)
else:
response = scraper.post(url, data=kwargs.get('data'), headers=headers, timeout=self.timeout)
response.raise_for_status()
return response
except ImportError:
raise Exception(Config.ERROR_MESSAGES["HTTP_CLIENT_NOT_AVAILABLE"].format(client="cloudscraper"))
elif client_type == "aiohttp":
try:
import asyncio
return asyncio.run(self._async_request(method, url, **kwargs))
except ImportError:
raise Exception(Config.ERROR_MESSAGES["HTTP_CLIENT_NOT_AVAILABLE"].format(client="aiohttp"))
raise Exception(Config.ERROR_MESSAGES["UNKNOWN_CLIENT_TYPE"].format(client_type=client_type))
async def _async_request(self, method: str, url: str, **kwargs):
"""Async HTTP request using aiohttp.
Args:
method: HTTP method (GET or POST)
url: Request URL
**kwargs: Additional request parameters
Returns:
MockResponse object with text attribute
"""
import aiohttp
headers = kwargs.get('headers', self.headers)
async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=self.timeout)) as session:
if method == "GET":
async with session.get(url, headers=headers) as response:
response.raise_for_status()
text = await response.text()
class MockResponse:
def __init__(self, text):
self.text = text
def raise_for_status(self):
pass
return MockResponse(text)
else:
async with session.post(url, data=kwargs.get('data'), headers=headers) as response:
response.raise_for_status()
text = await response.text()
class MockResponse:
def __init__(self, text):
self.text = text
def raise_for_status(self):
pass
return MockResponse(text)
def _is_404_error(self, error: Exception) -> bool:
"""Check if error is a 404 not found error.
Analyzes exception messages to determine if the error indicates
that the requested resource (app, developer, etc.) was not found.
Args:
error: Exception to check
Returns:
True if 404 error, False otherwise
Example:
if self._is_404_error(exception):
raise AppNotFoundError("App not found")
"""
error_str = str(error).lower()
# Check for common 404 error indicators
return "404" in error_str or "not found" in error_str
def _try_next_client(self):
"""Switch to next available HTTP client for retry.
Cycles through available HTTP clients when the current one fails,
providing automatic fallback functionality for improved reliability.
Process:
1. Move to next client in the list
2. Log the client switch
3. Reinitialize with new client
Note:
Called automatically by error handling decorators when retries are needed
"""
# Cycle to next available client
self.current_client_index = (self.current_client_index + 1) % len(self.available_clients)
next_client = self.available_clients[self.current_client_index]
logger.info(f"Switching to HTTP client: {next_client}")
# Update client type and reinitialize
self.client_type = next_client
self._setup_client()
def rate_limit(self):
"""Apply rate limiting delay between requests.
Implements rate limiting to prevent overwhelming the Google Play Store
servers and avoid getting blocked. Calculates the time since the last
request and sleeps if necessary to maintain the configured delay.
Rate Limiting Strategy:
- Tracks time of last request
- Enforces minimum delay between requests
- Prevents rapid-fire requests that could trigger blocks
Note:
Called automatically before each HTTP request
"""
current_time = time.time()
time_since_last = current_time - self.last_request_time
# Check if we need to wait before making the next request
if time_since_last < self.rate_limit_delay:
sleep_time = self.rate_limit_delay - time_since_last
logger.debug(Config.ERROR_MESSAGES["RATE_LIMIT_SLEEP"].format(sleep_time=sleep_time))
time.sleep(sleep_time)
# Update last request time
self.last_request_time = time.time()
================================================
FILE: output/app_example.json
================================================
{
"appId": "com.playdead.limbo.full",
"title": "LIMBO",
"summary": "Uncertain of his sister's fate, a boy enters LIMBO.",
"description": "What the press said: \n\n“Limbo is as close to perfect at what it does as a game can get.” \n10/10 – Destructoid \n\n“The game is a masterpiece.” \n5/5 – GiantBomb \n\n“Limbo is genius. Freaky, weird genius. Disturbing, uncomfortable genius.” \n5/5 – The Escapist \n\n“Dark, disturbing, yet eerily beautiful, Limbo is a world that deserves to be explored.” \n5/5 – Joystiq \n\nWinner of more than 100 awards, including: \n\nGameinformer’s “Best Downloadable” \nGamespot’s “Best Puzzle Game” \nKotaku’s “The Best Indie Game” \nGameReactor’s “Digital Game of the Year” \nSpike TV’s “Best Independent Game” \nX-Play’s “Best Downloadable Game” \nIGN’s “Best Horror Game”\n\nLimbo is an award-winning indie adventure, critically acclaimed for its captivating puzzle design and immersive sound and visuals. Its dark, misty spaces and haunting narrative will stay with you forever.",
"genre": "Adventure",
"genreId": "GAME_ADVENTURE",
"categories": [
"Adventure"
],
"available": true,
"released": "Feb 11, 2015",
"appAgeDays": 3898,
"lastUpdated": "Mar 11, 2025",
"updatedTimestamp": 1741708881,
"icon": "https://play-lh.googleusercontent.com/FJ8e7NYhyjzrjuROUSpigJ1TQNnZKUDh6AZc1SFjiD665bZsxr_7zus0DzlHIrC6Lgk=w9999",
"headerImage": "https://play-lh.googleusercontent.com/dPW585vpaYb9oNLGwCc9mrMc90NwUjzxYb-pJK07sUBhAmRib6DW5P3zSeA7DecMEw=w9999",
"screenshots": [
"https://play-lh.googleusercontent.com/GaptorFLFNZRTHSaV4Wh3R4nnhnd_LCCW1fNIwLERCyNcI3X5LOlK3TxeCKZLeoZowE=w9999",
"https://play-lh.googleusercontent.com/mAQtHG90gFvktB5AkGhqPqZEW6s-Ghqql4Jq6_Az4Y2hqOl7JN5oDG3MNHJJEEr4Rg=w9999",
"https://play-lh.googleusercontent.com/yxKadViR_JwU7Q5_s6OCEx3yU9WLR6Bo--kkkFThwUx5vNgxSPf3zbDDkjCWu_y0VQ=w9999",
"https://play-lh.googleusercontent.com/K1oLWUAPvUNSjbYQI4wHrDTwS2og0yDy5Hg2p2fCUlyOx9FchJzISBFyWj6Agip0Cms=w9999",
"https://play-lh.googleusercontent.com/Kj9lmcNbHpiaqJJLKvqwI9oPUKlO6Uqq0Alx5LjOZx7MrmCYA1JTQPtAO5FMpqUXcs2S=w9999",
"https://play-lh.googleusercontent.com/15Omhs7Wf3nNV0OL6SeAn7S-IL7_w0wRJjFcTWySrJWDXVnAWDvnepe44NtpdP4V-DRw=w9999",
"https://play-lh.googleusercontent.com/sMcfQtOUE_DgUOYY9vPVW6SCuAlwNSdbBcfmEMuNbjj9cFcAObSVV_06MOCMHNNJ_A=w9999",
"https://play-lh.googleusercontent.com/IJyIN5LH_WE9Wg8wurdKYH_mZZ3uZoc8qMIl4dcGAsorEDeSBnGGy7R9BW3VT_uwE4Q=w9999",
"https://play-lh.googleusercontent.com/zID8lHY-aSu8XwlNFbGeyYL9wHW4wcO3cpXb4nL6VzK404wE-Q164TL6a5uytSrbV0o=w9999",
"https://play-lh.googleusercontent.com/ofwJOH4PAJa1_-1YBopixmk1ps6A2Fk66yLj6TsW3aIEXLzLzhWoXuEKhyHc2kw72A=w9999",
"https://play-lh.googleusercontent.com/-72WGAXqv2DwytmhHKtXITNWD-nRrx1bsm7SFkckFFO0DoClhBxhukB0EhjBJ2bGDA=w9999",
"https://play-lh.googleusercontent.com/Ry0IjKLkNJpGCHHz17aW_dFa9geqgQyHJYyshAWhgzQCCQ55nUJgC8bkyE_TTxBxNQ=w9999",
"https://play-lh.googleusercontent.com/E-jL0foVeggCKVnuT8Y04jgnMZtpwxUqcPkEn29dDnI72rZk-r8N5yW5gW8tduLGFw=w9999",
"https://play-lh.googleusercontent.com/MBqe75pbbkCna_Nd3e8fa0nxKDwLrW17Q9o_y939DEXiiCxIQ_3DSxaRNto_AheG3XU=w9999",
"https://play-lh.googleusercontent.com/kBUx7Fp1gPWIyLpSo-VPf-o0inzbTUqQLpZ7mhCjLz4SR377SBCfDC4M3AZ78TRA2b4=w9999",
"https://play-lh.googleusercontent.com/_Uw03W-2-u3OnLPY7s5qaHzgdOPXGFezuXGHW9wQNbQSS82dzs-grm0VCBJPhcht-Q0O=w9999",
"https://play-lh.googleusercontent.com/xY4o2dmT4W8bVUthLgglH4IvZL-8Hn_7CSlgPbzLcJIuqdvJzcR7abQtN_TPxT1AvW0=w9999",
"https://play-lh.googleusercontent.com/-jMy1IndLzUj5YYaAMFyoeKALVNFgzH5giNhm4_45XcmOUXGXnr_1RNwjm_7q4X1g2vs=w9999",
"https://play-lh.googleusercontent.com/5sV41ilEro7ryUlt73TBfhnd5yqsY-Chv3_VxT2-qvRVbJM1E0vDYw5nK7Y_fvXh=w9999",
"https://play-lh.googleusercontent.com/Bd4BUBcKHDJCdPQ65jy-cVuZSldpIiCyXCprH3ZWjWGEMadCIf1j2VUd6_Q35luJ7zs=w9999",
"https://play-lh.googleusercontent.com/SZz2ehvolv0mNQBW6cZJ2Tm9mSVWl_S_2F2Kmojk8Kc-koWxOTyM5ZEx_Fd5uTo8Bi0=w9999",
"https://play-lh.googleusercontent.com/EgaZGAiGZ9fXz3BHY00j9-J32hLCRZ17tgG_LiJ-PPB2Kt160VOc_MT5wmWqma-y5GY=w9999",
"https://play-lh.googleusercontent.com/s3l_JhA1CbvglJO5XUN3Wlsfgc3c7BKI4xDaL8tjchPhU-Pb-uF9U82s8nOgvgi9KA=w9999",
"https://play-lh.googleusercontent.com/foTC5sz9VzPmFwsmDOZVHJL44LSwqcXHlgJdkPnKn38J-mOSr2KPXTvtZkSoDOaA4A=w9999"
],
"video": "https://play.google.com/video/lava/web/player/yt:movie:Y4HSyVXKYz8?autoplay=1&embed=play",
"videoImage": "https://play-lh.googleusercontent.com/dPW585vpaYb9oNLGwCc9mrMc90NwUjzxYb-pJK07sUBhAmRib6DW5P3zSeA7DecMEw=w9999",
"installs": "1,000,000+",
"minInstalls": 1000000,
"realInstalls": 3292879,
"dailyInstalls": 256,
"minDailyInstalls": 256,
"realDailyInstalls": 844,
"monthlyInstalls": 7809,
"minMonthlyInstalls": 7809,
"realMonthlyInstalls": 25714,
"score": 4.376488,
"ratings": 80722,
"reviews": 3899,
"histogram": [
6965,
2879,
2879,
8047,
59936
],
"adSupported": false,
"containsAds": false,
"version": "1.21",
"androidVersion": 7,
"maxAndroidApi": 35,
"minAndroidApi": 21,
"appBundle": "com.playdead.limbo.full",
"contentRating": "Teen",
"contentRatingDescription": null,
"whatsNew": [
"Bug fix for Snapdragon 8 Elite/Adreno 830 phones",
"Google API updates and crash fix",
"Update to packaging format (AAB)",
"Small icon update"
],
"permissions": {
"Photos/Media/Files": [
"read the contents of your USB storage"
],
"Storage": [
"read the contents of your USB storage"
],
"Other": [
"view network connections",
"Google Play license check"
]
},
"dataSafety": [
"No data shared with third parties",
"No data collected"
],
"price": 3.99,
"currency": "USD",
"free": false,
"offersIAP": false,
"inAppProductPrice": null,
"sale": false,
"originalPrice": null,
"developer": "Playdead",
"developerId": "9183662361966484245",
"developerEmail": "support@playdead.com",
"developerWebsite": "http://playdead.com",
"developerAddress": "Flæsketorvet 41\n1711 København V\nDenmark",
"developerPhone": "+45 53 76 41 00",
"privacyPolicy": "https://playdead.com/privacypolicy/index.php",
"appUrl": "https://play.google.com/store/apps/details?id=com.playdead.limbo.full"
}
================================================
FILE: output/developer_example.json
================================================
[
{
"appId": "com.google.android.apps.bard",
"title": "Google Gemini",
"description": "Supercharge your creativity and productivity with Gemini, your AI assistant from Google.\n\nGemini gives you direct access to Google’s best family of AI models on your phone so you can:\n\n- Go Live with Gemini to brainstorm ideas, simplify complex topics, and rehearse for important moments. Just click on the Gemini Live button in your Gemini app\n- Connect with your favorite Google apps like Search, YouTube, Google Maps, Gmail, and more\n- Study smarter and explore any topic with interactive visuals and real-world examples\n- Turn any file into a podcast that you can listen to anytime, anywhere\n- Create stunning images from just a few words\n- Plan trips better and faster\n- Get summaries, deep dives, and source links, all in one place\n- Brainstorm new ideas, or improve existing ones\n\nTry Nano Banana: state of the art image generation and editing built on Gemini 2.5 Flash\n\nLevel-up your Gemini app experience by upgrading to the Pro plan–unlock new and powerful features to tackle complex tasks and projects and enjoy industry-leading 1M token context window (enabling Gemini to process up to 1,500 pages of text or 30k lines of code), and:\n- Get more access our most powerful model, like 2.5 Pro\n- Generate and dive into detailed reports on any topic with Deep Research powered by 2.5 Pro\n- Turn words into high-quality, 8 second video clips with video generation with Veo 3, and more. \n\nGoogle AI Pro is available in 150 countries and territories, and includes additional benefits. Gemini app, as part of Google AI Pro, will continue to be available to qualifying Google Workspace business and education plans. Learn more: https://gemini.google/subscriptions/\n\nGet the best of Gemini app by upgrading to the Ultra plan–unlock the highest level of access and exclusive features to turn anything into anything. Get the highest access to Google’s most powerful model, like 2.5 Pro, and features like video generation with Veo 3 and Deep Research. You’ll also get early access to try our newest AI innovations as they become available, including Agent Mode.\n\nGemini in Google AI Ultra is available in the US and includes additional benefits as part of a Google AI Ultra subscription. Google AI Ultra is not currently available to Google Workspace business and education customers. Learn more: https://gemini.google/subscriptions/\n\nIf you opt in to the Gemini app, it will replace your Google Assistant as the primary assistant on the phone. Some Google Assistant voice features aren't available through the Gemini app yet. You can switch back to Google Assistant in settings.\n\nReview the Gemini Apps Privacy Notice:\nhttps://support.google.com/gemini?p=privacy_notice",
"icon": "https://play-lh.googleusercontent.com/bTpNtZ6rYYX2SeI-wC4cnr7MJnOh2hjtgYu3UIrSxE09lM3GPl_Uhf9_Ih2Smje2bc0V",
"developer": "Google LLC",
"score": 4.605983,
"scoreText": "4.6",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.bard"
},
{
"appId": "com.google.android.apps.youtube.unplugged",
"title": "YouTube TV: Live TV & more",
"description": "YouTube TV is now the exclusive home of NFL Sunday Ticket. Watch every out-of-market Sunday game* on your TV and supported devices. Learn more: https://yt.be/nflsundayticket\n\nWatch cable-free live TV. Download to watch & record live TV from 100+ networks, including local sports & news, as part of your monthly membership. Cancel anytime.\n\n+Cable-free live TV. No cable box required.\n+Stream major broadcast and cable networks, including ABC, CBS, FOX, NBC, NFL Network, ESPN, HGTV, TNT, AMC, and more, including your local sports & news channels.\n+Watch on your smartphone, tablet, computer, and TV\n+Cloud DVR without DVR storage space limits. Each recording will be stored for 9 months.\n+6 YouTube TV accounts per household. Everyone gets their own login, recommendations and DVR.\n+Monthly pay-as-you-go membership; cancel anytime.\n\nOver 100 networks are available in YouTube TV:\n\nBROADCAST\nABC, CBS, FOX, NBC, NFL Network, PBS, and more\n\nSPORTS\nCBS Sports Network, NBC Sports RSN (regional), NFL Network, ESPN, ESPN2, ESPNews, ESPNU, Golf Channel, NBA TV, SEC Network, and more\n\nENTERTAINMENT & LIFESTYLE\nAMC, Animal Planet, BBC America, BET, Bravo, Cheddar, CMT, Comedy Central, Comet, Cozi TV, Decades, Discovery, E!, Food Network, Freeform, FX, FXM, FXX, IFC, Investigation Discovery, HGTV, MotorTrend, MTV, Nat Geo, Nat Geo Wild, Oxygen, Paramount Network, Pop, Smithsonian Channel, SundanceTV, SyFy, TBS, TCM, TLC, TNT, Travel Channel, TruTV, TV Land, USA, VH1, WE tv, YouTube Originals, and more\n\nNEWS\nBBC World News, Cheddar Big News, CNBC, CNN, HLN, MSNBC, and more\n\nKIDS\nCartoon Network, Disney Channel, Disney Junior, Disney XD, Nickelodeon, PBS Kids, Universal Kids\n\n\nAvailability:\nYouTube TV is available nationwide in the United States.\n\nFor more information, please visit our Help Center.\n\nYour membership will automatically continue for as long as you choose to remain a member. Your membership is a month-to-month subscription that begins at sign up. You can easily cancel anytime, online, 24 hours a day. There are no long-term contracts or cancellation fees.\n\nSubscriptions automatically renew unless auto-renew is turned off at least 24-hours before the end of the current period. Account will be charged for renewal within 24-hours prior to the end of the current period.\n\nSubscriptions may be managed by the user and auto-renewal may be turned off by going to the user's Account Settings on the device.\n\nTerms of service: tv.youtube.com/tv/terms\nPaid terms of service: tv.youtube.com/tv/paidterms\nPrivacy policy: tv.youtube.com/tv/privacy\n\nEnjoy cable-free live tv now!\n\n*Commercial use excluded. Locally broadcast Fox and CBS games, Sunday Night Football on NBC, select digital-only games and international games excluded from NFL Sunday Ticket.",
"icon": "https://play-lh.googleusercontent.com/C-fxk_9e65qoZ9V9rb5uU8udyAnJU3IWnSldnoMqfFzk3wm4jCM9drsO2afVXGwXKyU",
"developer": "Google LLC",
"score": 3.9264245,
"scoreText": "3.9",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.youtube.unplugged"
},
{
"appId": "com.google.android.apps.translate",
"title": "Google Translate",
"description": "• Text translation: Translate between 108 languages by typing\n• Tap to Translate: Copy text in any app and tap the Google Translate icon to translate (all languages)\n• Offline: Translate with no internet connection (59 languages)\n• Instant camera translation: Translate text in images instantly by just pointing your camera (94 languages)\n• Photos: Take or import photos for higher quality translations (90 languages)\n• Conversations: Translate bilingual conversations on the fly (70 languages)\n• Handwriting: Draw text characters instead of typing (96 languages)\n• Phrasebook: Star and save translated words and phrases for future reference (all languages)\n• Cross-device syncing: Login to sync phrasebook between app and desktop\n• Transcribe: Continuously translate someone speaking a different language in near real-time (8 languages)\n\nTranslations between the following languages are supported:\nAfrikaans, Albanian, Amharic, Arabic, Armenian, Assamese, Aymara, Azerbaijani, Bambara, Basque, Belarusian, Bengali, Bhojpuri, Bosnian, Bulgarian, Catalan, Cebuano, Chichewa, Chinese (Simplified), Chinese (Traditional), Corsican, Croatian, Czech, Danish, Dhivehi, Dogri, Dutch, English, Esperanto, Estonian, Ewe, Filipino, Finnish, French, Frisian, Galician, Georgian, German, Greek, Guarani, Gujarati, Haitian Creole, Hausa, Hawaiian, Hebrew, Hindi, Hmong, Hungarian, Icelandic, Igbo, Ilocano, Indonesian, Irish, Italian, Japanese, Javanese, Kannada, Kazakh, Khmer, Kinyarwanda, Konkani, Korean, Krio, Kurdish (Kurmanji), Kurdish (Sorani), Kyrgyz, Lao, Latin, Latvian, Lingala, Lithuanian, Luganda, Luxembourgish, Macedonian, Maithili, Malagasy, Malay, Malayalam, Maltese, Maori, Marathi, Meiteilon (Manipuri), Mizo, Mongolian, Myanmar (Burmese), Nepali, Norwegian, Odia (Oriya), Oromo, Pashto, Persian, Polish, Portuguese, Punjabi, Quechua, Romanian, Russian, Samoan, Sanskrit, Scots Gaelic, Sepedi, Serbian, Sesotho, Shona, Sindhi, Sinhala, Slovak, Slovenian, Somali, Spanish, Sundanese, Swahili, Swedish, Tajik, Tamil, Tatar, Telugu, Thai, Tigrinya, Tsonga, Turkish, Turkmen, Twi, Ukrainian, Urdu, Uyghur, Uzbek, Vietnamese, Welsh, Xhosa, Yiddish, Yoruba, Zulu\n\nPermissions Notice\nGoogle Translate may ask for the following optional permissions*:\n• Microphone for speech translation\n• Camera for translating text via the camera\n• External storage for downloading offline translation data\n• Contacts for setup and management of your account\n\n*Note: The app may be used even if optional permissions are not granted.",
"icon": "https://play-lh.googleusercontent.com/ZrNeuKthBirZN7rrXPN1JmUbaG8ICy3kZSHt-WgSnREsJzo2txzCzjIoChlevMIQEA",
"developer": "Google LLC",
"score": 4.2877836,
"scoreText": "4.3",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.translate"
},
{
"appId": "com.google.android.apps.walletnfcrel",
"title": "Google Wallet",
"description": "Google Wallet gives you fast, secure access to your everyday essentials. Tap to pay everywhere Google Pay is accepted, board a flight, go to a movie, and more – all with just your phone. Keep everything protected in one place, no matter where you go.\n\nCONVENIENT\n\nGet what you need fast\n+ Three quick ways for accessing your everyday essentials: use your phone’s quick settings for fast access, open the Wallet app from your homescreen or use Google Assistant when your hands are busy.\n\nAccess Google Wallet from your Wear OS watch\n+ Get instant access to Wallet on the Wear OS main watch face with complications.\n\nCarry cards, tickets, passes, and more\n+ Catch a train, see a concert, or earn rewards at your favorite stores with a digital wallet that carries more\n+ [US Only] Unlock the world around you with a digital wallet that carries your drivers license and digital car keys\n\nWhat you need, right when you need it\n+ Your Wallet can suggest what you need, right when you need it. Get a notification for your boarding pass on the day of travel, so you’ll never have to fumble in your bag again.\n\nHELPFUL\n\nKeep track of receipts\n+ Easily find transaction details in Wallet, including smart details like location pulled from Google Maps\n\nSeamless integration across Google\n+ Sync your Wallet to keep your Calendar and Assistant up to date with the latest info like flight updates and event notifications\n+ Shop smarter by seeing your point balances and loyalty benefits in Maps, Shopping, and more\n\nGet started in a snap\n+ Set up is seamless with the ability to import cards, transit passes, loyalty cards and more that you’ve saved on Gmail.\n\nStay in the know on the go\n+ Make boarding flights a breeze with the latest information pulled from Google Search. Google Wallet can keep you posted on gate changes or unexpected flight delays.\n\nSAFE & PRIVATE\n\nA secure way to carry it all\n+ Security and privacy are built into every part of Google Wallet to keep all your essentials protected.\n\nAndroid security you can count on\n+ Keep your data and essentials secure with advanced Android security features like 2-Step Verification, Find My Phone, and remotely erasing data.\n\nTap to pay keeps your card secure\nALT: + When you tap to pay with your Android phone, Google Pay doesn’t share your real credit card number with the business, so your payment info stays safe.\n\nYou’re in control of your data\n+ Easy to use privacy controls allow you to opt-in to sharing information across Google products for a tailored experience.\n\nGoogle Wallet is available on all Android phones (Lollipop 5.0+), Wear OS and Fitbit devices.\nNot all features are available for supervised accounts. Learn more about Wallet for supervised accounts here: https://support.google.com/wallet?p=about_wallet_supervised.\nStill have questions? Head over to support.google.com/wallet.",
"icon": "https://play-lh.googleusercontent.com/DHBlQKvUNbopIS-VjQb3fUKQ_QH0Em-Q66AwG6LwD1Sach3lUvEWDb6hh8xNvKGmctU",
"developer": "Google LLC",
"score": 4.4905887,
"scoreText": "4.5",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.walletnfcrel"
},
{
"appId": "com.google.android.apps.youtube.kids",
"title": "YouTube Kids",
"description": "Inspire your kids to uncover their unique interests\nHelp your kids explore video content they love and parents trust, in an app made just for kids. With easy navigation tools and a suite of features, you can help your kids spend time online uncovering new interests, unleashing their imagination, and building their confidence in their own unique world.\n\nHelp your kids grow at their own pace\nYour kids are unique, so they should only see content they're ready to explore. Decide what videos will help them make the most of their time online, then personalize individual profiles using custom content filters as they grow.\n\n- Help your youngest kids learn their ABCs, nurture their curiosity, and more in the Preschool mode.\n- Expand your kids interests to songs, cartoons, or DIY crafts in Younger mode.\n- Give your older kids the freedom to search popular music and gaming videos in Older mode.\n- Or hand-pick the videos, channels, and collections your kids can see in Approved Content Only mode.\n\nRewatch videos and bond over favorites\nQuickly find your kids’ favorite videos and the content you’ve shared with them in the Watch it Again tab.\n\nShape your kids’ viewing experience with Parental Controls\nParental Control features help you limit what your kids watch and better guide their viewing experience. Our blocking process aims to help keep videos on YouTube Kids family-friendly and safe – but each family's preferences are unique. Don't like a video or channel, or see inappropriate content? Flag it for our team to review.\n\nSet a screen-time limit\nEncourage your kids to take a break in between exploring content. Use the Timer feature to freeze the app when screen time is up so your kids can apply their new skills to the real world.\n\nSee important information\n- Parental setup is needed to ensure the best experience for your family.\n- Kids may see commercial content from YouTube creators that are not paid ads.\n- See The Privacy Notice for Google Accounts managed with Family Link for information on our privacy practices for signing in with a Google Account.\n- If your kids use the app without signing in with their Google Account, the YouTube Kids Privacy Notice applies.",
"icon": "https://play-lh.googleusercontent.com/OxNGx8LU6gm8aLfJcwcJxunvj2a7zDgDyPOD4J9HRSIc6N_1O1iZ2dLr3xQMbuMy_wE",
"developer": "Google LLC",
"score": 4.1885204,
"scoreText": "4.2",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.youtube.kids"
},
{
"appId": "com.google.android.apps.authenticator2",
"title": "Google Authenticator",
"description": "Google Authenticator adds an extra layer of security to your online accounts by adding a second step of verification when you sign in. This means that in addition to your password, you'll also need to enter a code that is generated by the Google Authenticator app on your phone. The verification code can be generated by the Google Authenticator app on your phone, even if you don't have a network or cellular connection. \n* Sync your Authenticator codes to your Google Account and across your devices. This way, you can always access them even if you lose your phone. \n* Set up your Authenticator accounts automatically with a QR code. This is quick and easy, and it helps to ensure that your codes are set up correctly. \n* Support for multiple accounts. You can use the Authenticator app to manage multiple accounts, so you don't have to switch between apps every time you need to sign in. \n* Support for time-based and counter-based code generation. You can choose the type of code generation that best suits your needs. \n* Transfer accounts between devices with a QR code. This is a convenient way to move your accounts to a new device. \n* To use Google Authenticator with Google, you need to enable 2-Step Verification on your Google Account. To get started visit http://www.google.com/2step Permission notice: Camera: Needed to add accounts using QR codes",
"icon": "https://play-lh.googleusercontent.com/NntMALIH4odanPPYSqUOXsX8zy_giiK2olJiqkcxwFIOOspVrhMi9Miv6LYdRnKIg-3R",
"developer": "Google LLC",
"score": 3.7759008,
"scoreText": "3.8",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2"
},
{
"appId": "com.google.android.apps.adm",
"title": "Google’s Find Hub",
"description": "For your devices and items\n • View your phone, tablet, headphones, and other accessories on a map–even if they’re offline.\n • Play a sound to locate your lost device if it’s nearby.\n • If you’ve lost a device, you can remotely secure or erase it. You can also add a custom message to display on the lock screen in case someone finds your device.\n • All location data in the Find Hub network is encrypted. This location data is not visible even to Google.\n\n For location sharing\n • Share your live location to coordinate a meetup with a friend or check on family to make sure they got home safe.",
"icon": "https://play-lh.googleusercontent.com/GCsSBgR93cedwf2weP7s6VPsBitwir9ioOO0DYjLydIjdCkfQEv0GQzK34ky96L6XMc",
"developer": "Google LLC",
"score": 4.3170166,
"scoreText": "4.3",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.adm"
},
{
"appId": "com.google.android.apps.docs.editors.docs",
"title": "Google Docs",
"description": "Create, edit, and collaborate with others on documents from your Android phone or tablet with the Google Docs app. \n\nWork together in real time\n• Share documents with your team\n• Edit, comment, and add action items in real time\n\nCreate anywhere, anytime—even offline\n• Capture spontaneous ideas on the fly\n• Get things done, even on the go, with offline mode\n• Save time and add polish with easy-to-use templates\n\nEdit and share multiple file types\n• Open a variety of files, including Microsoft Word files, right in Google Docs\n• Frictionless collaboration, no matter which application your teammates use\n• Convert and export files seamlessly\n\nGoogle Docs is part of Google Workspace: where teams of any size can chat, create, and collaborate.\nGoogle Workspace paid subscribers have access to additional Google Docs features, including:\n• Use Gemini in Docs to quickly draft and edit content\n• Draft outlines, blog posts, briefs, and more on the go\n• Create unique images to customize your documents\n• Improve your writing with AI-powered suggestions\n\nLearn more about Google Docs: https://workspace.google.com/products/docs/\n\nFollow us for more:\n• X: https://x.com/googleworkspace\n• Linkedin: https://www.linkedin.com/showcase/googleworkspace\n• Facebook: https://www.facebook.com/googleworkspace",
"icon": "https://play-lh.googleusercontent.com/emmbClh_hm0WpWZqJ0X59B8Pz1mKoB9HVLkYMktxhGE6_-30SdGoa-BmYW73RJ8MGZQ",
"developer": "Google LLC",
"score": 4.2061143,
"scoreText": "4.2",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.docs.editors.docs"
},
{
"appId": "com.google.earth",
"title": "Google Earth",
"description": "Create and collaborate on immersive, data-driven maps from anywhere, with the new Google Earth. See the world from above with high-resolution satellite imagery, explore 3D terrain and buildings in hundreds of cities, and dive in to streets and neighborhoods with Street View's 360° perspectives.",
"icon": "https://play-lh.googleusercontent.com/9ORDOmn8l9dh-j4Sg3_S7CLcy0RRAI_wWt5jZtJOPztwnEkQ4y7mmGgoSYqbFR5jTc3m",
"developer": "Google LLC",
"score": 3.8520932,
"scoreText": "3.9",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.earth"
},
{
"appId": "com.google.android.apps.chromecast.app",
"title": "Google Home",
"description": "The Google Home app helps you get the most out of Gemini for Home.\n\n See your home at a glance \n The Google Home app is designed to show you the status of your home and keep you up to date with what you may have missed. \n\nKeep up with what’s important\nUpdated design and streamlined organization help you group your devices into dashboards and easily navigate your settings. Plus you can check in on your home anytime.\n\n Scan camera events quickly\n The camera live view and history interface makes it easier than ever to see what happened.\n\n Search or ask your home\n Control your home in a brand new way. Just say what you want your devices to do with Gemini for Home.\n\n* Some products and features may not be available in all regions. Compatible devices required.",
"icon": "https://play-lh.googleusercontent.com/-UJfyCGXOGNlA_7ys13IRTRxnrA5NGick84oQx-dBTr2swpki2xGIlHPcQaF46H21-u4",
"developer": "Google LLC",
"score": 4.159965,
"scoreText": "4.2",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.google.android.apps.chromecast.app"
}
]
================================================
FILE: output/list_example.json
================================================
[
{
"appId": "com.block.juggle",
"title": "Block Blast!",
"description": "Enter the world of Block Blast, a fun and free block puzzle game where every move challenges your brain and strategy, rewarding you with satisfying crush effects as you blast cubes away. Whether you're seeking offline relaxation or looking to enhance your logic skills, Block Blast delivers an engaging puzzle experience.\n\n🌟 Why You'll Love Block Blast\n🔸 Vibrant Puzzle Adventure: Match colorful blocks on an 8x8 board, solve logic puzzles, and watch the cubes crush in a cascade of color.\n🔹 Strategic Combos & Streaks: \nUse your brain to clear multiple lines of blocks in one move for powerful combos. Maintain your streaks to achieve massive scores!\n🔸 Casual Yet Challenging: Play at your own pace, relax your brain, or apply deep logic strategies to beat your highest score.\n🔹 Offline Fun, No WiFi Needed: Enjoy free block puzzle game anytime and anywhere, perfect for endless fun on-the-go.\n\n💥 Features of Block Blast\n● Puzzle Gameplay: Strategically place blocks and cubes, match colors, and solve each challenging level.\n● Adventure Mode: Take on progressively challenging puzzles, each level with unique theme jigsaw and visual styles.\n● Daily Challenges: Keep your brain sharp with daily logic puzzles and earn exclusive achievements.\n● Optimized for All Devices: Smooth, seamless gameplay experience with minimal memory usage—perfect for phones and tablets.\n\n🎮 How to Play\n● Drag & Match: Strategically place cubes on the 8x8 board.\n● Crush & Score: Complete rows or columns to crush blocks away and earn points.\n● Chase Combos: Plan your moves and trigger massive combos for bonus points.\n● Stay Strategic: Apply logic and strategy to prevent your board from filling up.\n\n✨ Pro Tips for Puzzle Masters\n● Plan Ahead: Visualize your moves to keep the board open for larger cubes.\n● Maximize Combos: Match multiple rows and blocks simultaneously to boost your combos.\n● Master the Streak: Consistent clears will maintain your fun streaks and enhance your score rewards.\n\n🔥 Jump into the Free Block Puzzle Game!\nReady to test your brain, sharpen your logic, and enjoy hours of offline puzzle fun? Download Block Blast now and embark on an unforgettable block puzzle adventure.",
"icon": "https://play-lh.googleusercontent.com/R0qgNDYYHbRhw6JFsdEbDMqONplEvJx0m0W9wzYVvY3eNF1c2rfBWYjQxW0sLEzFe1E",
"screenshots": [
"https://play-lh.googleusercontent.com/JH136Ry9BOxHs9cIpcw5yo7A5UaSsNJz9Ovj_vqqytRjJuSPtEZTF51dtpyJtZcxdg",
"https://play-lh.googleusercontent.com/Iu9-uepyJdBINl618OEI4SGUoA1rj0QzUGPlhY855UDDNppNZ3J77CXIQn14imoWuw",
"https://play-lh.googleusercontent.com/wXreIezMLn2ozcR1ENWNIynZaxG3MbY2OjZDN6Rd868uN-09grdxmFdMLChhEaGyY3Y",
"https://play-lh.googleusercontent.com/PY0e_rJe6FlDmvBq8TVTUP3leYptQkXotS4doXK9vzbVatXJ6kYkQcJbVPQjYOCNVco",
"https://play-lh.googleusercontent.com/EjrWRONeRO8EYhV6tWb2cyb3wRYWqbDxy_B8vVqCsnJTJ3hMeceI0wdESsyip9lTrkU",
"https://play-lh.googleusercontent.com/hE9WNCB4Rxsa_H938nAuWk8AuLV4agpgKWO93vRsy3UT7_XGmSn0Alwidb6ue9xTYKo",
"https://play-lh.googleusercontent.com/k0so2CHbsaTGhMQL8jc0r68y9LAQu_zH6nrsFRp3-DSah3d1dpLt3l3hsa8n36kmWBA",
"https://play-lh.googleusercontent.com/bRBFNo-YL5jw2Pqg6xfg64AQoMpJgzRpZC2bJ0hGPVpDu_j1jByZnTBUDpak-kb4M9U",
"https://play-lh.googleusercontent.com/mwc7--BK6xtanBvV1nDXiIs0LfmK6eLavzQR3yf51UyNjDfFDHn9H_KueRcB99IQC9Y",
"https://play-lh.googleusercontent.com/ohIoigvvKwLfKHRclTAUYzgtwGWuk8EvoA7QdqH8GVtQ3vIT-DBtQpn95c0w0S_gxO8j",
"https://play-lh.googleusercontent.com/7mJin-3Gov7TEjaNAAboEEXqpZCdI1I8MOlAVht55WO1qS2MxmhFakmNerQaB4xXBQU",
"https://play-lh.googleusercontent.com/8GGwXesJ2MlKRvxnPdIHk5R8CjIR7BscsYk_vDIeVqY7bI1Mh4BeIxy62zEKkuTzMVI",
"https://play-lh.googleusercontent.com/DpK-36oHtNE0aKFY928s5SWKkbc0AHfp8AFAT1t2ByEczaTruknrM6lqdPc_7yAb__4",
"https://play-lh.googleusercontent.com/V_L-yGwRnrPOg5WNvKooDioYnlzFW24EkXO71MyyRt2OaQPW6V12uVzApqFY9r-OpQ",
"https://play-lh.googleusercontent.com/YxbhB1CWMdpVJdmUKfLx1VkJBbKl8kcUbMxplDBHFE4S0-BWA2hpQnfSTwoux4eig4g",
"https://play-lh.googleusercontent.com/RpTpLs0KyvlkgTfxxV0Mb1OmuSa9ztMvEdxQvCIPPROO0nNPA50O8hEfctPl2cZXkXJA",
"https://play-lh.googleusercontent.com/IFKlsGcUwVpsk0ZC8bdqyLazRfC9dsmNjM8GGLFNUaMIPi27A5u0hT2po9LgQnJ1aDY",
"https://play-lh.googleusercontent.com/kpa6YPXIhsHfpHhuIp8zcqv4DsN-l5g_JuTMzCLyFwfChkd9CjrlbOg0fWvSiS-PZZU",
"https://play-lh.googleusercontent.com/mu3aB1q44fGu-Dw8zChd5jA_BPbIp8j0JHwq48szrJ1LXftLbxmkW08ZZM0_LBgBAdQ",
"https://play-lh.googleusercontent.com/N6y3WtmTuEbkHgIsU-kqiVkWSC5JAmDeUplJGflPjn4BLYSUUBHQVQJgufCDb7bCa6vj",
"https://play-lh.googleusercontent.com/KQwgw62ch32NPRON-GQmsZfvbyEtaJ3zgKLCkeQ9NHJAbkOETQUIpcT7-tQ4xSh22Y8",
"https://play-lh.googleusercontent.com/pghY7dgd8j6orApxNpbmf9H4f9uEHgyFG0-EeWRRcWex2sovygRDoKVQqvDn0wRjfTc",
"https://play-lh.googleusercontent.com/MNBGZQp3syHpVfc2VP35Hg0H6GFXOC3h-BQOl9_IHOLOmy55tfdoK5o4p0cacvuSpQ",
"https://play-lh.googleusercontent.com/FY5ZkuQ-sxW97T9XDlXouB0JWKOlwe2FsGhxyAEWrY8JyJjCTXcioDcEhaHzWza1wdk"
],
"developer": "HungryStudio",
"genre": "Puzzle",
"score": 4.845834,
"scoreText": "4.8",
"installs": "500,000,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.block.juggle"
},
{
"appId": "com.gameberry.sorry.card.board.game",
"title": "Sorry! World - Board game",
"description": "Sorry! is Online Now\n\nNow you can enjoy the classic Sorry! game online for free with Sorry World, a digital adaptation of Hasbro's popular board game. \n\nSorry World features pawns, a game board, a modified deck of cards, and a designated Home zone. The goal is to move all of your pawns across the board into the Home zone, which is a safe area. The player who successfully gets all of their pawns Home first is the winner.\n\nHow To Play\n\nSorry World is a family-friendly board game for 2 to 4 players where the goal is to move all three of your pawns from Start to Home before your opponents. \nHere's how to play:\n\n1. Setup: Each player selects a color and places their three pawns in the Start area. Shuffle the deck of cards and place it face down.\n\n2. Objective: The first player to move all three of their pawns around the board and into their Home space wins the game.\n\n3. Starting: Players take turns drawing a card from the deck and move their pawns according to the card’s instructions. The deck includes cards that allow players to move forward, backward, or swap places with an opponent.\n\n4. Sorry Card: Drawing a \"Sorry!\" card lets you replace any opponent's pawn on the board with one of your own, sending their pawn back to Start.\n\n5. Landing on Opponents: If you land on a space occupied by another player's pawn, that pawn is bumped back to Start.\n\n6. Safety Zones and Home: Pawns must enter their Home space by exact count, and the final stretch leading to Home is a \"safe zone\" where opponents can’t bump you out.\n\nSorry World combines strategy, luck, and opportunities to foil opponents’ plans, making each game competitive and exciting.\n\nSorry World is a fun, free to play online board game. It is very similar to Ludo, Parcheesi, like board games.",
"icon": "https://play-lh.googleusercontent.com/rr_m94Psm7I9MB83viB-MSn3Sh4p_ZmVCkLBZFLKx9SQIkbE0sb7K9WJx2cuYVf-OCk",
"screenshots": [
"https://play-lh.googleusercontent.com/WUEznddfz88QtmOd2Dbhvs8Ox5mwdO4PsIKK9_TahQN5haxAKPeUXCNnGZY1MvjoGOty3tM3FlFDJQa2kA2W",
"https://play-lh.googleusercontent.com/CvK50Kp7ED_SKDKHJnodtbAZwoGyFuW1WP4EaMu2AF5LpIq1H-PhOM5tGVXb-YRe_A33BQNyWPB8yn_-xe2bGQ",
"https://play-lh.googleusercontent.com/nImu-z3SVbbzQ1vcl8FoAMlqWHBWq7fgy1spc7T16JxLwnOlGGcXughyxgvq08pphpB172itvbtN3Z3vMU-i0A",
"https://play-lh.googleusercontent.com/WBGxESr1IPf393lIOLG9yjGDVhkwtoP8Sm8IkMWB5OfFrsFASQ5B9Dz679ehPRqa4TNpDG2AwvRUvDv_a0ioIsU",
"https://play-lh.googleusercontent.com/kv8G4djWURJ5Mqc1P8CAxGJDrLUjHD2CsfqbZhyiFdWRxRASZfydSi8F0MVRYz2Nvm42u6do4mycBhUbGl2B",
"https://play-lh.googleusercontent.com/MiZO1fgcS_BtvrSdBgTc_2pJvGCuSa7RWT_7fmlolIFVrrgboiNe0ztl4mLbdpQp4b4mcA7POyfVciyDOS2JEA",
"https://play-lh.googleusercontent.com/yHn2U48tWNYs-IatnQV-iVn1X4p7x5EvpfFE_Ez2Ons2HjwFXwgo4RgjEcogx6kjvbGbDBNb1WM5_a2L_YY9vg",
"https://play-lh.googleusercontent.com/ylr42iSdiBp-LrjfKzvhI1JJKzu3shbKtH_7NSu-MlcTSwH9Nd-pG-lF-4WrTJAUKXn91bJlhVKTO2whbYVNAA",
"https://play-lh.googleusercontent.com/WUEznddfz88QtmOd2Dbhvs8Ox5mwdO4PsIKK9_TahQN5haxAKPeUXCNnGZY1MvjoGOty3tM3FlFDJQa2kA2W",
"https://play-lh.googleusercontent.com/CvK50Kp7ED_SKDKHJnodtbAZwoGyFuW1WP4EaMu2AF5LpIq1H-PhOM5tGVXb-YRe_A33BQNyWPB8yn_-xe2bGQ",
"https://play-lh.googleusercontent.com/nImu-z3SVbbzQ1vcl8FoAMlqWHBWq7fgy1spc7T16JxLwnOlGGcXughyxgvq08pphpB172itvbtN3Z3vMU-i0A",
"https://play-lh.googleusercontent.com/WBGxESr1IPf393lIOLG9yjGDVhkwtoP8Sm8IkMWB5OfFrsFASQ5B9Dz679ehPRqa4TNpDG2AwvRUvDv_a0ioIsU",
"https://play-lh.googleusercontent.com/kv8G4djWURJ5Mqc1P8CAxGJDrLUjHD2CsfqbZhyiFdWRxRASZfydSi8F0MVRYz2Nvm42u6do4mycBhUbGl2B",
"https://play-lh.googleusercontent.com/MiZO1fgcS_BtvrSdBgTc_2pJvGCuSa7RWT_7fmlolIFVrrgboiNe0ztl4mLbdpQp4b4mcA7POyfVciyDOS2JEA",
"https://play-lh.googleusercontent.com/yHn2U48tWNYs-IatnQV-iVn1X4p7x5EvpfFE_Ez2Ons2HjwFXwgo4RgjEcogx6kjvbGbDBNb1WM5_a2L_YY9vg",
"https://play-lh.googleusercontent.com/ylr42iSdiBp-LrjfKzvhI1JJKzu3shbKtH_7NSu-MlcTSwH9Nd-pG-lF-4WrTJAUKXn91bJlhVKTO2whbYVNAA",
"https://play-lh.googleusercontent.com/WUEznddfz88QtmOd2Dbhvs8Ox5mwdO4PsIKK9_TahQN5haxAKPeUXCNnGZY1MvjoGOty3tM3FlFDJQa2kA2W",
"https://play-lh.googleusercontent.com/CvK50Kp7ED_SKDKHJnodtbAZwoGyFuW1WP4EaMu2AF5LpIq1H-PhOM5tGVXb-YRe_A33BQNyWPB8yn_-xe2bGQ",
"https://play-lh.googleusercontent.com/nImu-z3SVbbzQ1vcl8FoAMlqWHBWq7fgy1spc7T16JxLwnOlGGcXughyxgvq08pphpB172itvbtN3Z3vMU-i0A",
"https://play-lh.googleusercontent.com/WBGxESr1IPf393lIOLG9yjGDVhkwtoP8Sm8IkMWB5OfFrsFASQ5B9Dz679ehPRqa4TNpDG2AwvRUvDv_a0ioIsU",
"https://play-lh.googleusercontent.com/kv8G4djWURJ5Mqc1P8CAxGJDrLUjHD2CsfqbZhyiFdWRxRASZfydSi8F0MVRYz2Nvm42u6do4mycBhUbGl2B",
"https://play-lh.googleusercontent.com/MiZO1fgcS_BtvrSdBgTc_2pJvGCuSa7RWT_7fmlolIFVrrgboiNe0ztl4mLbdpQp4b4mcA7POyfVciyDOS2JEA",
"https://play-lh.googleusercontent.com/yHn2U48tWNYs-IatnQV-iVn1X4p7x5EvpfFE_Ez2Ons2HjwFXwgo4RgjEcogx6kjvbGbDBNb1WM5_a2L_YY9vg",
"https://play-lh.googleusercontent.com/ylr42iSdiBp-LrjfKzvhI1JJKzu3shbKtH_7NSu-MlcTSwH9Nd-pG-lF-4WrTJAUKXn91bJlhVKTO2whbYVNAA"
],
"developer": "Gameberry Labs",
"genre": "Board",
"score": 4.61758,
"scoreText": "4.6",
"installs": "500,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.gameberry.sorry.card.board.game"
},
{
"appId": "com.roblox.client",
"title": "Roblox",
"description": "Roblox is the ultimate virtual universe that lets you create, share experiences with friends, and be anything you can imagine. Join millions of people and discover an infinite variety of immersive experiences created by a global community!\n\nAlready have an account? Log in with your existing Roblox account and explore the infinite metaverse of Roblox.\n\nMILLIONS OF EXPERIENCES\n\nIn the mood for an epic adventure? Want to compete against rivals worldwide? Or do you just want to hang out and chat with your friends online? A growing library of experiences created by the community means there’s always something new and exciting for you every day.\n\nEXPLORE TOGETHER ANYTIME, ANYWHERE\n\nTake the fun on the go. Roblox features full cross-platform support, meaning you can join your friends and millions of other people on their computers, mobile devices, Xbox One, or VR headsets.\n\nBE ANYTHING YOU CAN IMAGINE\n\nBe creative and show off your unique style! Customize your avatar with tons of hats, shirts, faces, gear, and more. With an ever-expanding catalog of items, there’s no limit to the looks you can create.\n\nCHAT WITH PEOPLE YOU KNOW\n\nParty is a seamless way for up to six friends to group up and jump into an experience together. Join people you know and stay together as you move across experiences. 13+ users can also use Party to chat through voice or text. It's never been easier to coordinate and communicate on Roblox.\n\nCREATE YOUR OWN EXPERIENCES: https://www.roblox.com/develop\nSUPPORT: https://en.help.roblox.com/hc/en-us\nCONTACT: https://corp.roblox.com/contact/\nPRIVACY POLICY: https://www.roblox.com/info/privacy\nPARENT’S GUIDE: https://corp.roblox.com/parents/\nTERMS OF USE: https://en.help.roblox.com/hc/en-us/articles/115004647846\n\nPLEASE NOTE: A network connection is required to join. Roblox works best over Wi-Fi.",
"icon": "https://play-lh.googleusercontent.com/7cIIPlWm4m7AGqVpEsIfyL-HW4cQla4ucXnfalMft1TMIYQIlf2vqgmthlZgbNAQoaQ",
"screenshots": [
"https://play-lh.googleusercontent.com/qUcj9ZD2zwc43jbDwpF2BpGPE6PeVKvrJYPaPdYBf9Hn3MSjZcDvKVSFTWlNT0Q75J2ur-rlSxKSud5fjm6Y_yQ",
"https://play-lh.googleusercontent.com/WkH4eVcLSN3Wt9FE2wA5Us_FKsZkQag_XKVij50MWUgCS4lcxMl8Qhz5KHwUeREVBQB49QQTlFVra-grTWA",
"https://play-lh.googleusercontent.com/DWcwhYgqQeWlyMaIzkWjdty00t4otXy6qcCRVsTT-weay9uqhnNYM9x_127dA92vWTY2ujxFRo2UQIFxHZlWRmk",
"https://play-lh.googleusercontent.com/CtOd6lRcb_4EpkF6fXdabp2wuUssau6a2e_oBebJs3_xDBeTFLO9rPikfEO6pUdnO5un75_OviGepFo6zk3FNm8",
"https://play-lh.googleusercontent.com/c5Ix5Ct0yy0Jv3QqYA1MWSp5sSW_8OlnWRYWwximhHLDlBmTwteJAi_pl51DYPV_wZK0E5YEukRC7pQFLfQE",
"https://play-lh.googleusercontent.com/qUcj9ZD2zwc43jbDwpF2BpGPE6PeVKvrJYPaPdYBf9Hn3MSjZcDvKVSFTWlNT0Q75J2ur-rlSxKSud5fjm6Y_yQ",
"https://play-lh.googleusercontent.com/WkH4eVcLSN3Wt9FE2wA5Us_FKsZkQag_XKVij50MWUgCS4lcxMl8Qhz5KHwUeREVBQB49QQTlFVra-grTWA",
"https://play-lh.googleusercontent.com/DWcwhYgqQeWlyMaIzkWjdty00t4otXy6qcCRVsTT-weay9uqhnNYM9x_127dA92vWTY2ujxFRo2UQIFxHZlWRmk",
"https://play-lh.googleusercontent.com/CtOd6lRcb_4EpkF6fXdabp2wuUssau6a2e_oBebJs3_xDBeTFLO9rPikfEO6pUdnO5un75_OviGepFo6zk3FNm8",
"https://play-lh.googleusercontent.com/c5Ix5Ct0yy0Jv3QqYA1MWSp5sSW_8OlnWRYWwximhHLDlBmTwteJAi_pl51DYPV_wZK0E5YEukRC7pQFLfQE",
"https://play-lh.googleusercontent.com/qUcj9ZD2zwc43jbDwpF2BpGPE6PeVKvrJYPaPdYBf9Hn3MSjZcDvKVSFTWlNT0Q75J2ur-rlSxKSud5fjm6Y_yQ",
"https://play-lh.googleusercontent.com/WkH4eVcLSN3Wt9FE2wA5Us_FKsZkQag_XKVij50MWUgCS4lcxMl8Qhz5KHwUeREVBQB49QQTlFVra-grTWA",
"https://play-lh.googleusercontent.com/DWcwhYgqQeWlyMaIzkWjdty00t4otXy6qcCRVsTT-weay9uqhnNYM9x_127dA92vWTY2ujxFRo2UQIFxHZlWRmk",
"https://play-lh.googleusercontent.com/CtOd6lRcb_4EpkF6fXdabp2wuUssau6a2e_oBebJs3_xDBeTFLO9rPikfEO6pUdnO5un75_OviGepFo6zk3FNm8",
"https://play-lh.googleusercontent.com/c5Ix5Ct0yy0Jv3QqYA1MWSp5sSW_8OlnWRYWwximhHLDlBmTwteJAi_pl51DYPV_wZK0E5YEukRC7pQFLfQE",
"https://play-lh.googleusercontent.com/qUcj9ZD2zwc43jbDwpF2BpGPE6PeVKvrJYPaPdYBf9Hn3MSjZcDvKVSFTWlNT0Q75J2ur-rlSxKSud5fjm6Y_yQ",
"https://play-lh.googleusercontent.com/WkH4eVcLSN3Wt9FE2wA5Us_FKsZkQag_XKVij50MWUgCS4lcxMl8Qhz5KHwUeREVBQB49QQTlFVra-grTWA",
"https://play-lh.googleusercontent.com/DWcwhYgqQeWlyMaIzkWjdty00t4otXy6qcCRVsTT-weay9uqhnNYM9x_127dA92vWTY2ujxFRo2UQIFxHZlWRmk",
"https://play-lh.googleusercontent.com/CtOd6lRcb_4EpkF6fXdabp2wuUssau6a2e_oBebJs3_xDBeTFLO9rPikfEO6pUdnO5un75_OviGepFo6zk3FNm8",
"https://play-lh.googleusercontent.com/c5Ix5Ct0yy0Jv3QqYA1MWSp5sSW_8OlnWRYWwximhHLDlBmTwteJAi_pl51DYPV_wZK0E5YEukRC7pQFLfQE"
],
"developer": "Roblox Corporation",
"genre": "Adventure",
"score": 4.417539,
"scoreText": "4.4",
"installs": "1,000,000,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.roblox.client"
},
{
"appId": "com.dreamgames.royalkingdom",
"title": "Royal Kingdom",
"description": "From the creators of Royal Match comes a brand new match 3 puzzle adventure in Royal Kingdom, starring the extended Royal Family!\n\nYou’ll meet King Richard, King Robert's younger brother, as well as a charming cast of new characters, including Princess Bella and the Wizard, to embark on a journey to build legendary kingdoms! Solve match 3 puzzles to explore new lands and defeat the Dark King & his army!\n\nMASTER MATCH 3 PUZZLES\nTest your skills and become the ultimate match 3 expert by solving fun yet challenging puzzles! Beat thrilling levels and overcome unique obstacles!\n\nBUILD AND EXPLORE KINGDOMS\nWith the help of the Builder, craft a kingdom befitting royalty. Solve puzzles, earn coins, and unlock diverse districts - from the Parliament Square to the University and the Princess Tower.\n\nCONQUER THE DARK KING\nDefend the kingdom from the Dark King’s attack by solving match 3 puzzles - destroy his castles and evil minions to see him fall. Victory is one match away!\n\nEXPAND YOUR RULING\nRise through the ranks and claim the top spot on the leaderboard, master your puzzle solving skills for generous rewards, and expand your kingdom by uncovering uncharted lands as you play!\n\nENJOY THE FINEST VISUALS\nImmerse yourself in Royal Kingdom's stunning graphics and smooth animations. A puzzle game experience like never before - captivating and seamless.\n\nWhat are you waiting for? Download Royal Kingdom and join the ranks of the noble adventurers! With hours of fun, challenging gameplay, and a magical world, this puzzle game is fit for royalty!",
"icon": "https://play-lh.googleusercontent.com/nAAF2bxrD2BeoM1IkUZ6qeRG_QGG_Rekrrh4VYQSMDeRa7FR8sWMWaHClID547XJ3D4",
"screenshots": [
"https://play-lh.googleusercontent.com/GVMQQ6FoNEuF4cJJNk2Rn6Tgagc4zvI8L7_YmBazeLvoiTcrAdLsHjo5QJlr0QJL2G68",
"https://play-lh.googleusercontent.com/R0kpGTgMTYqsZr4ivwVlkRHFXGXdaVyiLJ7mZvTWlNdSBQB3mMpSM-V2XIXl2xacYtY4",
"https://play-lh.googleusercontent.com/xsoi6B61S3SpYnF035o1bdijETm9E_Ff-Z-mqUH76jmGovj_7Za-7z5B3FSRweMPPY4",
"https://play-lh.googleusercontent.com/vbXGl-v3lQRefMG5PwT1bGeWSe0MYBDtSzD7OO24cnZRMmzmFo5u4Q85bBivzJqNbMpH",
"https://play-lh.googleusercontent.com/Tm86QGILOzisu1czGXBnpOIXbrf-Yt11R09EmkEiqRXmYGS6XdWp7uXCIgWxLLc6nw",
"https://play-lh.googleusercontent.com/g28RpdvTloLnI9uO4hZgcSKD58Rs7tSjv3CpwwwisciuEhHJq5ADPy4Ao00PxKL05h5p",
"https://play-lh.googleusercontent.com/R3j0OQDK5ybQssm4gTzSrPLhrTk-y1KDJjqC3ipOutd5Feo1yrnE6vnYXPinre4f7w",
"https://play-lh.googleusercontent.com/ll3r1Nic53TS6xQo-KfHr7Mx1UklrXpywpQuJ-l672QVpMFYWfAc34K692rpAnjO_bon",
"https://play-lh.googleusercontent.com/vJz9J6bHwrONLFBIvtC8u3Imkw4hNlpLlZrbbOpQPW6OG0BVZFTc4KITBD3TBQyWpSE",
"https://play-lh.googleusercontent.com/tTF-7VQIFEgQnW_dlijwIr1f4oltoV3I-_H7gRv8SSaU-uds6-SKMF-2dH_i3wrU2Q",
"https://play-lh.googleusercontent.com/ItehiqS4Mw6K_cQsKVkNXqn3C6DA8SH3oBVPOC3-Lnlhu0y85VbtqfjygQvNPR0bNIdd",
"https://play-lh.googleusercontent.com/IU5-2OPurC0IezgkRqBJ1JhqatPVeBHVik1XL4gPw1joyqfd-FQxzJGtdI2R4lELiw",
"https://play-lh.googleusercontent.com/Vl-BjUFcI_AEzH9oP-94Nzxs3nQfMeVyfUiDbL8lFRo7Jw4Se2YWZZakvgyYTGWAmUI",
"https://play-lh.googleusercontent.com/TuGo_ZwivBos_X2aUJ2scwu0Jux_rpczRIEfRoEvub_1_9H6CfIimycVWmr2gK7wKrI",
"https://play-lh.googleusercontent.com/83sNVPSUjvASONLwNg_Kt_UPcJY_FxEqZGGFz69qpEIRmKR-AkkRhQmAZckGdMhRrOg",
"https://play-lh.googleusercontent.com/OjgLAF7x6BPS_le-YbewdeHhC-iWxdWBg5hQLSS0SSgfgTfBI4mfgrJETpRKZtIZcrs",
"https://play-lh.googleusercontent.com/mm3eq8gnqU75nxIKumYi7_1tk21FlmbYtuXsTOgiU3AapX34PrAItfzME8g4eBtYMSc",
"https://play-lh.googleusercontent.com/jOyXPjIN0FWP3SOH98Yi2YcfSKNH_3mqOMWEqwH-haZJqYqQv_vD2DePP95ArZMiWWbT",
"https://play-lh.googleusercontent.com/VnE7CTLw-LJNDGYux1YKDRcuGX-5u5I7KzTfZOwcFzQaxi5vQb__k_4JPq8n7LgFipfZ",
"https://play-lh.googleusercontent.com/2Xcmz31MStvxfty8Zz7JbnFqdKgwOjo4B0O2UhfYudrBGF01tkM0swlzpCURMHZOTXsj",
"https://play-lh.googleusercontent.com/IBfFxYssBlzlw4D9FqErfLjz5i9vXbVF6wbaYOAVm0tavwiKF7ggDOI6XE0RMjXcJTA"
],
"developer": "Dream Games, Ltd.",
"genre": "Puzzle",
"score": 4.629104,
"scoreText": "4.6",
"installs": "10,000,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.dreamgames.royalkingdom"
},
{
"appId": "com.tfgco.apps.coloring.free.color.by.number",
"title": "Color by Number：Coloring Games",
"description": "🎨 Discover a world of relaxation, creativity, and diversity with Color by Number: Coloring Game - the ultimate stress-relieving pixel art game! Explore our vast collection of color by number images from various cultures and artists worldwide or import your own pictures to create a personalized color by number experience. Join millions of users and share your masterpieces with our vibrant community.\n\n🌟 Key Features:\n• Easy color by number: Dive into a wide variety of pixel art images designed for both beginners and experienced colorists, featuring popular themes and unique content.\n• Create unique images: Import pictures from your gallery or snap a photo to turn your memories into color by number masterpieces.\n• Community sharing: Share your color by number creations and discover other users' pixel art in our thriving community of coloring enthusiasts.\n• Diverse painting tools: Experiment with different tools to make coloring and drawing more fun and efficient.\n• Inclusive and diverse art: Immerse yourself in a rich variety of color by number art from different cultures and artists, promoting creativity and inclusiveness.\n• Antistress and relaxation: Enjoy the calming effects of art therapy as you unwind with our engaging and satisfying color by number game.\n\n🖌️ Get ready to relax and express your creativity with this free coloring game that combines the joy of art with the benefits of stress relief. Whether you're a seasoned artist or just looking for a peaceful pastime, Color by Number: Coloring Game offers an engaging and satisfying experience for everyone. Download now and start your colorful journey today!",
"icon": "https://play-lh.googleusercontent.com/7BIu-nLPuxUMTDATJ_mZu1wVMZAaxMGjJuQFrGxUS7-pb4IXZqGRq8LKJEXzJrejB3Tf",
"screenshots": [
"https://play-lh.googleusercontent.com/iLsVJPqCXBrQexaEgePloGuwmHFeXA_C6eNwHdiUh9GnAWY0EHyeYyxcIkQQ4YfqSXuW",
"https://play-lh.googleusercontent.com/ZfqyCfBt77R_6mtBjDthtKcqFK6gbDv0GaPMvSV7Ghosr1miLdt4C9eT-0vZ8X5zDKU",
"https://play-lh.googleusercontent.com/xfWILkgd_NR3FOh_6Kk6hsPPE6-DrwoCPkieYB5j5SUj7xQt3WsCf36-19C87RIeFw",
"https://play-lh.googleusercontent.com/38UtJpiKd2StyNnWqmaZL3BFMDXCyfwILczdDkmFjpAv_X4fIR7Y_SxoXXx9l_M5s8uh",
"https://play-lh.googleusercontent.com/BWSmX9LCO13zVDwsIztA0z61cNymwmuGmAFm0yKZ1pnhlse4nJ2Z_l0z1bPLboKT_g",
"https://play-lh.googleusercontent.com/KjcL67-nw7RkfHENmoA0YzEVi0m4uSo5a1bBhq_-GHbMJZMJNCDEku6crY-gRqcQGnGC",
"https://play-lh.googleusercontent.com/pcPJlKTDDCBJ-3mywRGZ-2h0E6rHPU0OcQedx5kTBnFzrAbLa6waMh_We5pimSV5xNa0",
"https://play-lh.googleusercontent.com/tnd2cISvUULqLPYLdxC1ladXxHYZqQ0U9O6_IY0lMHwLjrMv1WJ_m5jVfcv2T00nMRb-",
"https://play-lh.googleusercontent.com/AF5HnTkbCczub7OsV6J4ubQLg08Jq5juaYet7kW_XFKHdp_7oQInn3LIcXYLULeFohfK",
"https://play-lh.googleusercontent.com/Knsw9oZVTYvaeNEXsRvc7db2KrW925bUeUws_ZqvIGRfDDPRMNd-dGtoJCcQm3aRli8r",
"https://play-lh.googleusercontent.com/Xk4MVHqiqpaNwwj5EIYp9-GaF8yrET0Ob3lI1LO73JpmqEwcqyvH1lYYThfEG91V5K8",
"https://play-lh.googleusercontent.com/dNilqHW9D3ZYUTwG9OvwecSIhvjL5gVKQZDNpdmumymReg2CnTpYE31o0FShdqQ4OwE",
"https://play-lh.googleusercontent.com/OSp2zlX6cJfm6dxnwArXg0wdbL2IHM5Jvvt0ukEe_-t_AXIkH7F2qfRQwl9zJkHo6ho",
"https://play-lh.googleusercontent.com/7-e4VD3Jj6z5vwf8F4vWuITM0Mun-cQz0NuoLw53Lp5DofmzSzDJiIKi4KnN0RnBJMU",
"https://play-lh.googleusercontent.com/yaB56aFLGCrGC9HUTSRHrjRlOQzsLVMBfKhW3tMgRvn5N-B-MFC1vjbNRWUUulo47bo",
"https://play-lh.googleusercontent.com/T5vaIf8H7qsBVlF_JkJW8LzeFjvICkyjMB240oMdTQ0XjGXtDwrYoMWE6Hk0nD6Qi9M",
"https://play-lh.googleusercontent.com/B0ue2cfyYB_sjKCeRG-lmlm0UTy2U8Jp4An0-YPvbXlWGeazlcEE7AjY8IElwOSVV-4",
"https://play-lh.googleusercontent.com/55BLcOJh2rQE3HYCzl9-LYrAWo-PCIgjYk2zLVDxQfdKxabO9MModtKhR6i1Tfin5Pc",
"https://play-lh.googleusercontent.com/BIuj-TOEXm4zOf-3QQDNwgUoh_A2KMd7onLEpXlgk52OTDrKGU8Zg1s1U9rU24ZOuw",
"https://play-lh.googleusercontent.com/PMg3SGh1t3eVlmUYW2-b0vOlQBqRb6r-_x1PHuJQq-EgE6j_9fmVlRsIRD8wbgYYuQ",
"https://play-lh.googleusercontent.com/LPZ7C5hDUfbIuDwBC1vqzKgDOAw88pfzGpsklI2GZRYbRUiw36qrWW34x7jp90hB9Rg",
"https://play-lh.googleusercontent.com/jefU3f4ZliV4IQzcBCeGaLrkEpE-2PJRcxuKHEoaVkA17c7aINs1Gl1SwjY-WqWvxA",
"https://play-lh.googleusercontent.com/mxlJtS05Uv3JU-vnTcmbuq6b3p7ODePrJmRQHzMG3KX3pd1Wn1LANTwFDQ_BnotZBhXO",
"https://play-lh.googleusercontent.com/JEV9d-M5b52WluoY2Q-XCuEzH5xwHpkGpEncsaq23xhCX3m3lep11WDl8qyelPcN9kY"
],
"developer": "Wildlife Studios",
"genre": "Board",
"score": 4.662116,
"scoreText": "4.7",
"installs": "50,000,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.tfgco.apps.coloring.free.color.by.number"
},
{
"appId": "com.jamcity.pdt",
"title": "Disney Magic Match 3D",
"description": "Dive into the enchanting world of Disney Magic Match 3D, a captivating puzzle adventure with beloved Disney treasures! Disney Magic Match 3D is a brand-new 3D matching game where you’ll solve captivating Disney puzzles by sorting and collecting iconic items from beloved Disney and Pixar films including Moana, Aladdin and Toy Story. The ancient Disney Book of Magic, filled with iconic artifacts, has been opened, releasing a whirlwind of nostalgia. It's your mission to return these cherished items to their rightful place by sorting and matching the pieces in magical puzzles. \n\nDISCOVER ICONIC 3D DISNEY TREASURES!\n\nExplore stunning 3D levels inspired by your favorite Disney films and discover a vast array of nostalgic items. From Mickey Mouse's iconic gloves to Aladdin's magic lamp or Woody’s boots from Toy Story, each 3D piece is brimming with Disney charm. Unlock new chapters in the magic book and uncover even more Disney treasures in Enchanted Levels! Enchanted Levels include themed pieces from your favorite films or nostalgic characters, featuring classics like Mickey Mouse’s shoes, Minnie Mouse’s bow, and Donald Duck’s hat. \n\nSORT AND MATCH IN ICONIC DISNEY PUZZLES!\n\nTest your skills with a variety of engaging Disney puzzles that will challenge your mind and invoke nostalgia. Match and sort scattered Disney items from the Heart of Te Fiti to a vast array of princess dresses. By identifying and sorting scattered items, each completed match brings you closer to restoring order to the universe. Feel the satisfaction of sorting the treasures back into their correct places in Disney Magic Match 3D! \n\nDownload Disney Magic Match 3D and experience enchanting Disney puzzle levels with Aladdin, Toy Story, Moana and more!",
"icon": "https://play-lh.googleusercontent.com/nJlQmDZ8r33Gr9DY04eREaokemxlsy3SA59kFrkTzy3cyq_xUUX6Qt6AhqWIR-QNCycJ0KwquGPPO7KsLNLzKMQ",
"screenshots": [
"https://play-lh.googleusercontent.com/nKBxYzDNM2WqGBaZFJV3Mk8Ki-qb4YVzkcs5ONAsMkDHX0FhMSXRClwZqAUveQC0owhpqDtBmzSSTD0CQ4FzOg",
"https://play-lh.googleusercontent.com/Dbh2WnpaDXxo8JIOSF1OUu6W3ZeMibRZmVozUtCW1P5TAx8DuQ22Px9ogzBu6kT5-HJQiAVjElYvbFJUmevzoUY",
"https://play-lh.googleusercontent.com/BzNObgKE1bPjzaHekXEOPaBN1Z91mkYCo_3OTNG9CZwMbRNk4Jk2brUPrnPO9yVl9kX8xdsQf-jNd8xQJr5mO1M",
"https://play-lh.googleusercontent.com/p7H30il41lJc28JkqpZdLpnZ4H4fM5g_R6pNUB-FYLTVZ7RhG7Os996rJF1phAO_AyojFIzXIQYBcCvrUP-tcm4",
"https://play-lh.googleusercontent.com/7CD3AEJE2kVwxVOiKAZaBjr0-UPFoTCjALZvyQOQRdIjNqcppKz0C5esgtMhg3GwJnJIK8BcpdAiLITg9SC6jQ",
"https://play-lh.googleusercontent.com/DX8g_26YI8L2ebLEdVtHain5E3UDjbwM5zd9wofgsWzVxh_uU__MVrzUSLoPHzbiEw",
"https://play-lh.googleusercontent.com/nKBxYzDNM2WqGBaZFJV3Mk8Ki-qb4YVzkcs5ONAsMkDHX0FhMSXRClwZqAUveQC0owhpqDtBmzSSTD0CQ4FzOg",
"https://play-lh.googleusercontent.com/Dbh2WnpaDXxo8JIOSF1OUu6W3ZeMibRZmVozUtCW1P5TAx8DuQ22Px9ogzBu6kT5-HJQiAVjElYvbFJUmevzoUY",
"https://play-lh.googleusercontent.com/BzNObgKE1bPjzaHekXEOPaBN1Z91mkYCo_3OTNG9CZwMbRNk4Jk2brUPrnPO9yVl9kX8xdsQf-jNd8xQJr5mO1M",
"https://play-lh.googleusercontent.com/p7H30il41lJc28JkqpZdLpnZ4H4fM5g_R6pNUB-FYLTVZ7RhG7Os996rJF1phAO_AyojFIzXIQYBcCvrUP-tcm4",
"https://play-lh.googleusercontent.com/7CD3AEJE2kVwxVOiKAZaBjr0-UPFoTCjALZvyQOQRdIjNqcppKz0C5esgtMhg3GwJnJIK8BcpdAiLITg9SC6jQ",
"https://play-lh.googleusercontent.com/PRbZBM1C2o86ZnmjZQydtNoBLYX4cjciATPWfjqq5eN34P-t2Bgcvtb4NAbrgJ_znw",
"https://play-lh.googleusercontent.com/nKBxYzDNM2WqGBaZFJV3Mk8Ki-qb4YVzkcs5ONAsMkDHX0FhMSXRClwZqAUveQC0owhpqDtBmzSSTD0CQ4FzOg",
"https://play-lh.googleusercontent.com/Dbh2WnpaDXxo8JIOSF1OUu6W3ZeMibRZmVozUtCW1P5TAx8DuQ22Px9ogzBu6kT5-HJQiAVjElYvbFJUmevzoUY",
"https://play-lh.googleusercontent.com/BzNObgKE1bPjzaHekXEOPaBN1Z91mkYCo_3OTNG9CZwMbRNk4Jk2brUPrnPO9yVl9kX8xdsQf-jNd8xQJr5mO1M",
"https://play-lh.googleusercontent.com/p7H30il41lJc28JkqpZdLpnZ4H4fM5g_R6pNUB-FYLTVZ7RhG7Os996rJF1phAO_AyojFIzXIQYBcCvrUP-tcm4",
"https://play-lh.googleusercontent.com/7CD3AEJE2kVwxVOiKAZaBjr0-UPFoTCjALZvyQOQRdIjNqcppKz0C5esgtMhg3GwJnJIK8BcpdAiLITg9SC6jQ",
"https://play-lh.googleusercontent.com/2xrFn5QMt39xAKYUMi5_B1CVhKh9-q60GDEyWmW0P51K6sZsDjDxMqxYuhEY_z49ww"
],
"developer": "Jam City, Inc.",
"genre": "Puzzle",
"score": 4.751634,
"scoreText": "4.8",
"installs": "500,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.jamcity.pdt"
},
{
"appId": "com.supercell.clashroyale",
"title": "Clash Royale",
"description": "Enter the Arena! Build your Battle Deck and outsmart the enemy in fast real-time PvP tower defense card games. From the creators of CLASH OF CLANS comes a real-time multiplayer card battle game starring your favourite Clash® characters and more. Start battling against players from around the world!\n\nBECOME A MASTER OF STRATEGY, TOWER DEFENSE AND DECK BUILDING\nChoose unique Cards for your Battle Deck and head to the Arena for multiplayer PvP strategy games!\nPlace your Cards right and knock down the enemy King and Princesses from their Tower defenses in strategic, fast-paced matches.\n\nCOLLECT AND UPGRADE 100+ CARDS\nHog Rider! Collect and upgrade 100+ Cards featuring the Clash of Clans troops, spells and defences you know and love. Win multiplayer PvP card battle games and progress to new Arenas to unlock powerful new Cards for your collection!\n\nBATTLE YOUR WAY TO THE TOP\nStrengthen your tower defense, fine-tune your strategy and card battle your way to the League games and Global Tournaments! Match against the best players in the world and compete in multiplayer PvP battles for glory and rewards!\n\nSEASONAL EVENTS\nUnlock new Seasonal items like Tower Skins, Emotes and powerful Magic Items with the Season Pass and participate in fun Challenges that put your card battle and tower defense skills to the test!\n\nJOIN A CLAN AND GO TO WAR\nJoin or form a Clan with other players to share Cards, and battle in multiplayer Clan Wars card games for BIG rewards!\n\nSee you in the Arena!\n\nPLEASE NOTE! Clash Royale is free to download and play, however, some game items can also be purchased for real money. If you do not want to use this feature, please set up password protection for purchases in the settings of your Google Play Store app. Also, under our Terms of Service and Privacy Policy, you must be at least 13 years of age to play or download Clash Royale.\n\nA network connection is also required.\n\nSupport\nAre you having problems? Visit http://supercell.helpshift.com/a/clash-royale/ or http://supr.cl/ClashRoyaleForum or contact us in game by going to Settings > Help and Support.\n\nPrivacy Policy:\nhttp://supercell.com/en/privacy-policy/\n\nTerms of Service:\nhttp://supercell.com/en/terms-of-service/\n\nParent’s Guide:\nhttp://supercell.com/en/parents/",
"icon": "https://play-lh.googleusercontent.com/gnSC6s8-6Tjc4uhvDW7nfrSJxpbhllzYhgX8y374N1LYvWBStn2YhozS9XXaz1T_Pi2q",
"screenshots": [
"https://play-lh.googleusercontent.com/UOJ0N42bDu2lUbZIx4n9UCnHtnY5IEyG1jOLXByCbbCvi6wammxVR4XC9endWA5rAA",
"https://play-lh.googleusercontent.com/-H9tX-JiL-u169fW5aPKBOO2kIO20HIPdLcuSE-VKH3UPzhg5225q6OPy-If0pu_c1o",
"https://play-lh.googleusercontent.com/KJyB5262htBywP40LmNgt3WCr0yGjXDkiaY2t1T3FL71Ph_Dk5iYqd5UJC6NIaUbf-w",
"https://play-lh.googleusercontent.com/4xvy3712rgooE8zG-b3ePxPXU5eWFa78q57TClvKQcHtEn9yFVHq_By3do9wqzyy18Ig",
"https://play-lh.googleusercontent.com/KMU1IyCAbHzfNVzihvyedCDXDW3uIpKbaWNj2BPSxhrTyCnzWH88lRUIX2BnMyRD888",
"https://play-lh.googleusercontent.com/ve38p_0sUn23Be3ou6p1BLJXuhrADDN97V7hTr_7Hdvj5-LyMusyaRHqn_oEMHqClKo",
"https://play-lh.googleusercontent.com/4spxVWDXk34CQGpAXvyp2GRhOuPhJquP21iGUwgbl0QbPisEMU_waQGFttSNkety4SQ",
"https://play-lh.googleusercontent.com/HKLKrTnr3COplkpdKN5tsdDKz0K38EOQUQAI1EuQBOOCpFIN7rGhRQwdkTI0VAA8c_8",
"https://play-lh.googleusercontent.com/0lxyuOHmLJ0nU446lyJxMB-YPqhnHAtcfXBAzD3cdg_L2jtr0-19IDKKtZQfzWUho_M",
"https://play-lh.googleusercontent.com/QsLLNH_rDEpTLJplnyIHNlyRShGDEnL8bziYdxsP55moVU-SzQ-iSq-ZWJz7CSPoRt27",
"https://play-lh.googleusercontent.com/QJ9ykyt6fihGGdYjs6SRqAx2kK3KMqpNoxL80ibkRu2t9HqiApaGvyPy6AmgdQK9MIQ",
"https://play-lh.googleusercontent.com/fzxOuYpsjRyHzm81NZUHstQsJEKBTcAm4UrFiAcBtf1fJAyTRFudFXXQBlWpGIeXPzA",
"https://play-lh.googleusercontent.com/Ze81cW27HS-ABTlyyT0Y1XjN3fv-wVXHtAPvdYTOBqptV6DuLzGR7-xVZnGjr3HLGA3T",
"https://play-lh.googleusercontent.com/XRXQjoDacywQQs66Ea1rbywlZ9hoq8pvigIxQhI_UMQQITfLN9CTH-6YcGubz1OQ3_jO",
"https://play-lh.googleusercontent.com/BTSNh21ZUoXvEXpAlVhXCsnKpk_8BCaa4ofU0Q9DxNR-pERChD7TpTw0GXPrat7mPA",
"https://play-lh.googleusercontent.com/vPmwD3jbwKj2W_xYiKgoV7wLmoqnceDVHVE6r6AoZQpBlolCkRNRG_0ZgMJfySQLKg",
"https://play-lh.googleusercontent.com/OTIPaWEQ2STGiqlh7bCTnIUtXliKYVBu1JN6A1z31xjlCKmNnpbM1ZR6J_vNOn1zNzTb",
"https://play-lh.googleusercontent.com/5VYLM_j_5sFd0pqL8j95Ms8Jkvd4RTE1jzqQPmcRRpRNCfFmP6-Dh-men3apSlDlfWA"
],
"developer": "Supercell",
"genre": "Strategy",
"score": 4.435455,
"scoreText": "4.4",
"installs": "500,000,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.supercell.clashroyale"
},
{
"appId": "com.kiloo.subwaysurf",
"title": "Subway Surfers",
"description": "DASH as fast as you can! \nDODGE the oncoming trains! \n\nHelp Jake, Tricky & Fresh escape from the grumpy Guard and his dog. \n\n★ Grind trains with your cool crew! \n★ Colorful and vivid HD graphics! \n★ Hoverboard Surfing! \n★ Paint powered jetpack! \n★ Lightning fast swipe acrobatics! \n★ Challenge and help your friends! \n\nJoin the most daring chase! \n\nA Universal App with HD optimized graphics.\n\nOriginally co-developed by SYBO and Kiloo.",
"icon": "https://play-lh.googleusercontent.com/2j-z3t20D8yLBLScuzp3sFfLCR40PHmQxdVO-O2JrFtcDA7pmg0ln-MmEDcm7tUsLn4",
"screenshots": [
"https://play-lh.googleusercontent.com/aRRu8u4--Ssh6rtXEeZYKvc-mK6x_xoAOssFP_v0JJrGZp0VgO9jXWEh2bDyZT30Ww",
"https://play-lh.googleusercontent.com/gwQSt5GSRzvmnAW3-W3h8YCV295XbVg3iijuTHiYKGViPSb97Fk3Qd_4WJbe5CcU35iR",
"https://play-lh.googleusercontent.com/Gu-RYFUPkwbeCbfHa1hrrEqIO8SMOIpvRMiw8ELnar1zjmda__2VmqPkVT_VtKxHBA",
"https://play-lh.googleusercontent.com/ecqSKEAsKcGvcszd6g28h3qBYzqpHp_6DPet-sURQBOmFVK6cg30LXPKr2wpPafEvBQ",
"https://play-lh.googleusercontent.com/OSHM1XAxe1z8UYAn57p5YPTv-6i8C3sahAToA80ywjMaU46XAz1fybb0I7r0eB3Fcr0",
"https://play-lh.googleusercontent.com/QYENgJVrNTT8zAFGiRvnxcdULnVlhLtnijKP8ALLRVhJ_ISRYOhgjK-LtJF_aSU6CQ",
"https://play-lh.googleusercontent.com/L6R-9z8rMZzTCFMeQnwKX1VP1hHfVy9zFQXszgm2agR1IO-ulSKdYngM3MyC91a_dM0",
"https://play-lh.googleusercontent.com/e384H7-gEctPIPFKDj4BODEvWYKP60efW18crVFv5QoDKd2zQWZcFSrhIqTUIRvetVU",
"https://play-lh.googleusercontent.com/osgNROHm5FqAIMv30Ss4lfkFaKJSrxYGO0dhxIsCPksnsGQoKYHdRJorTf0VCWP0Tzc",
"https://play-lh.googleusercontent.com/pp9XR7B6NXVXzi17sSMpCR-SIgx8BQh1No79Y2njSzbKQ_0aX1PnEVmP_L9TEuUfYKe2",
"https://play-lh.googleusercontent.com/5NBH4_1q8VVuqePC_nXlhRN78wBJANSZvu-VnsR-OZkK_26Q4oUTv5r9JlqzGiWWsuiU",
"https://play-lh.googleusercontent.com/BM5MraXpqcSHjMrPIjl17qdkAI9bfp-8Og9FdgsDlrmWbCNNSOZKoiuzAlBDkAbcQbM",
"https://play-lh.googleusercontent.com/PAoDaUr7QJPCXSluO6-INdqGySnhR-Wrgou6An7wGhYdLA-hsAQ35LTpYKb74OsqMQA",
"https://play-lh.googleusercontent.com/w6ePMD1wpQQazdud2ES1u6O2KHuH0qbH6UQpiStmsIGqzMrcywORNeX4K-6EnJxhJVr5",
"https://play-lh.googleusercontent.com/qgoitoEsf5j3JcazVxx3fNpBDId7ZsSRJJ_xVjf4GySe7Kanf6vEdNTteLilP_YywuaK",
"https://play-lh.googleusercontent.com/hBi_b1JHC5keSM5-tDekL6F-IKl5vr03jC8ERffMnjzPn0R6b_N9vUmYCEN51sxX-Q",
"https://play-lh.googleusercontent.com/dtTDm-sy2hzhSOTojaqWUbXLhUYpB3UmXKN04pFqSezREptORcP28cl0fi3kMxQGRNc",
"https://play-lh.googleusercontent.com/3CgA6eyTNGmkFGii0OVjtTawnSYB3klgVN8sXoXGVtEvBozM3HYPsDVj-kLFlrI5rOkr",
"https://play-lh.googleusercontent.com/9tjEiO2gr6kwvdzpLEaZvlDqvdjZvMvLyXctmRhePLHVXE6a6zRdq2aTy9mSqOInDqI",
"https://play-lh.googleusercontent.com/a-cT5PeAAQgX1HnrYrhT6mZf7JgV3R9xGz1_ey9KUiiR0Qro8dpdNleHvJ6OCAf6nR8",
"https://play-lh.googleusercontent.com/3ie8HMeVTyjTL9ACFrssPuKK9kvPImof-6HN5rTzF_uGkbhCrNjcpejvUpVbWmaZ0M8",
"https://play-lh.googleusercontent.com/4oLzN8lDSbSQmvspQqrL00uKD501B2HqfTvk9kUzb_q13XgJ7bs99sEfU3a-9F-Xy7JA",
"https://play-lh.googleusercontent.com/8wIDXrpESoPfAhiDxBQXaFuzeeRe9Hrm9Ablu9yiixQwOxpdS9bnmAD79xj2Czy1W8s",
"https://play-lh.googleusercontent.com/H6hk3ZusKbJOjppMrDd-mTDVtbgl6-Ik9Yz6qq6RzYhUh-4Q24yhUAcslarR4byO5fA"
],
"developer": "SYBO Games",
"genre": "Arcade",
"score": 4.5551095,
"scoreText": "4.6",
"installs": "1,000,000,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.kiloo.subwaysurf"
},
{
"appId": "com.scopely.monopolygo",
"title": "MONOPOLY GO!",
"description": "Hit GO! Roll the dice! Earn MONOPOLY money, interact with your friends, family members and fellow Tycoons from around the world as you explore the expanding universe of MONOPOLY GO! It’s the new way to play the classic board game - board flipping cleanup not required in these fun board games!\n\nTake a Break!\nEscape, enjoy, dream, scheme, gain rewards and stay in touch with this newly reimagined twist on MONOPOLY online games! Let everyone’s favorite zillionaire, Mr. MONOPOLY, guide you through new boards themed after world-famous cities, fantastical lands and imaginative locales. Each adventure begins with a roll of the dice - whether you're a board game beginner or one of the strategy masters, there's fun to be had for everyone!\n\nSo MONOPOLY GO!\n· Experience classic fun and visuals with gameplay fit for your phone! Collect Properties, build Houses and Hotels, pull Chance Cards, and of course, roll the dice to earn that MONOPOLY Money!\n· Play with your favorite game Tokens such as the Racecar, Top Hat, Battleship, and more. Earn more tokens as you go to keep the win going in these board games!\n· See classic MONOPOLY icons like Mr. M, Scottie and Ms. MONOPOLY come to life, and brand new characters too!\n\nYour Family Table!\n· Help or hinder! - Play dice & pick a card - You and friends can earn easy money with Community Chest and co-op events! Or heist their banks to help yourself get to the top. Don’t feel bad – that’s just how the dice roll! \n· Collect and trade story-filled Stickers with friends and family around the world and in our MONOPOLY GO game! Facebook Trading Groups! Complete gorgeous, sticker albums to win huge rewards! The more Stickers you collect, the closer you get to candy sweet exclusive bonuses and limited-edition collections!\n\nFeatures!\n\nBUY & BUILD YOUR WAY TO THE TOP\nCollect Property Tile Sets to build Houses and upgrade your Houses to Hotels to get even more rent from friends! All you have to do is hit GO and roll the dice! Build your empire and be lord of the board! Whether you're in it for casual fun or to prove you're one of the true masters of the game – the board awaits!\n\nENJOY THAT CLASSIC MONOPOLY ATMOSPHERE\nRoll the dice to enjoy the classic game of MONOPOLY. Featuring familiar faces such as MR. MONOPOLY, familiar spaces such as jail (womp womp!), Railroads, Properties, Tokens, and familiar elements like drawing the perfect lucky card and more!\n\nPLAY WITH FRIENDS AND FAMILY\nGet social! With a variety of fun games, you can play with friends around the world to take full advantage of new multiplayer mini-games such as Community Chest – where you and friends take a break from mischief and work together for fun and rewards!\n\nNEW OPPORTUNITIES EVERY DAY\nPlay Tournaments, the Prize Drop plinko mini-game, the Cash Grab mini-game and follow our Events for big rewards. New Events run every hour, there are new ways to play and win every day! Compete in every tournament to climb the ranks. Keep an eye out for our time-limited games, perfect to challenge even seasoned tournament champions and MONOPOLY masters alike. Every roll of the dice counts - will it land you bonus money, a valuable Sticker, or a big build upgrade in new lands from a candy factory to Martian land?!\n\nMONOPOLY GO! is free to play, though some in-game items can also be purchased for real money. Internet connection is required to play the game.\n\nThe MONOPOLY name and logo, the distinctive design of the game board, the four corner squares, the MR. MONOPOLY name and character, as well as each of the distinctive elements of board and playing pieces are trademarks of Hasbro, Inc. for its property trading game and game equipment. © 1935, 2023 Hasbro.\n\nPrivacy Policy: https://scopely.com/privacy/\n\nTerms of Service: http://scopely.com/tos/\n\nAdditional Information, Rights, and Choices Available to California Players: https:scopely.com/privacy/#additionalinfo-california\n \nBy installing this game you agree to the terms of the license agreements.",
"icon": "https://play-lh.googleusercontent.com/DfYkSl-nQoMNLX2bec7EwHemrvyDYmDgzIR1jcsyt0ZAcmO_SKjuu0a1o1iSwtnl8_g",
"screenshots": [
"https://play-lh.googleusercontent.com/WNKQ6fTz9CGyaCRGu7s-Bte4RMpblAruUx_RE7QQ1OmaywMASbYg0MBxRZu-42njpRTd",
"https://play-lh.googleusercontent.com/rr1qJMrWD-dKQhPef-XR9v-Sy_hiyXsvr0fauB3McVoK_k7kwn8otr5v8xEjkSElN30",
"https://play-lh.googleusercontent.com/btLf4QGAB24NJYON3S1dipSO6qmBrrBy1g0KjX-5PPpTXX6fJbPMAYLXHtx_Rca4jww",
"https://play-lh.googleusercontent.com/v5r7Wb7sq11nMFl4WEjkeSGd3mnti_gCmoQPzI6WJ0vYjllGhmVu9qF_Pn_vQtyvxQGW",
"https://play-lh.googleusercontent.com/wJoBNVNl0dryHECtUV8B-BlddxVcd76YkJwJ1y7KIfDH-fYNOHlCx1TzA7hFx6y_rB0",
"https://play-lh.googleusercontent.com/M-XKIP-T-UyJd3IyTPTon7RFDUTm1qNNlu7bUmkqwuycFyg1LthPZ1aVwsuKvqgFnRU",
"https://play-lh.googleusercontent.com/UfaRoz251VtP8-5wGw16GhTkopL47rqmOzoBoQexcZyxsCgALF4q6W9ycLhBFmcCoBk",
"https://play-lh.googleusercontent.com/UbcoF0euH86amf-EKoLaocRG8bAb601pN3ODrW6Zsa3KMwNbqcJ8yP3uJf_yT_QHVg",
"https://play-lh.googleusercontent.com/ZRds25BCinCMjFE6KPXBsHPxeF2qf33XXhCPYbep63dvKxGoEhxBJiwASq7p1kTh2ahz",
"https://play-lh.googleusercontent.com/iuGZbyGeS-8wZi_xw1B1d3A8zTaEj1SxvXns0n-yD5hojYItDcw4OBt_6v2HCkqMQw",
"https://play-lh.googleusercontent.com/D2kHrhJgWsGxquJjCRy_whhx_k4gsPEf6hhzVvP0Slb6lyZqmfXHOH4StHbSndPGwg0",
"https://play-lh.googleusercontent.com/n_cHy8m8Bv_xYJ4kZ3p0Yk5kuWCzw-ypbCtmWropBfUhMhwyVJULs1HngfLNAf3FcLI",
"https://play-lh.googleusercontent.com/_OZxTwMrIv8H_yGtvn1e6Q46vs0w8KeRrDGay_nzYy62Yx6omLmeuK6X9CKHxkt9mFs",
"https://play-lh.googleusercontent.com/uyGSikeWJmOnLtStNucCN2k25THwO6hVo7dyMVVTCjo5z6COzAOefPIsJyQdxVojS20",
"https://play-lh.googleusercontent.com/nhfYCYHZrXLlVCZCZHuVvejuHBgs40OnWfWnRRumjAqjdJVrBOQLU3TzIpg_xTS5jKU",
"https://play-lh.googleusercontent.com/6Se8VezTdt2XRz4wxMdwSv18DYhH8B3dgQvGU5euhnBP3ggdlAM5SSkP6EzE2e0JPg",
"https://play-lh.googleusercontent.com/9KWlnQCIHz1m7IERSEPDWYAx3J8oNUQ47yfqv7MOdyrKqQ68rNJJtgyiWvKxm85OEEU",
"https://play-lh.googleusercontent.com/7kSLZ_MwqXQpdxw8GwFcnTyD6IIK_VwJOOnU3RpthD_nOS8rhyrGSBTf6PJZR_Aib-s",
"https://play-lh.googleusercontent.com/t4flDlmdC1rUJbNZAEAtSku6ZzGk0K9xZEFkjKAhD2avEYEm1VXjVTnq3_3Y7GNTBw",
"https://play-lh.googleusercontent.com/7GfvHqDJMvtS4cMhjVQGGvEwhC6oOdI6PNx_PUNDVw8SZWAoFmLt3XLhYzTfMdOttg",
"https://play-lh.googleusercontent.com/eHA0WcX8Q8aHjM5_1idtwlOYGN5QqAglO9_QmHFjDEOKl0BUGy_51WBqnbwHiDJgU9M",
"https://play-lh.googleusercontent.com/orPNnERkLcQc1gH2y_X41r2CkBHJ7C4Xo60LD8zj8mIIcT-HoswT35V6dVpVskwHtNE",
"https://play-lh.googleusercontent.com/JTDnVbve9Scy9BubgXTGl8MiWHuO4t_H2Eqqb-qHBi5DseIICcwgPtZS3D4d3LED_kM",
"https://play-lh.googleusercontent.com/8zODvpEX2n9W2lfAJ-g0oYLLthFjVkNr6jJhZNxzDTHeCEvQxSkUTsbe6IymrZtsvA"
],
"developer": "Scopely",
"genre": "Board",
"score": 4.6633234,
"scoreText": "4.7",
"installs": "100,000,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.scopely.monopolygo"
},
{
"appId": "io.voodoo.holeio",
"title": "Hole.io",
"description": "Hole.io – Swallow Everything & Dominate the City!\n\nEnter the ultimate black hole battle and compete to become the biggest hole in town! Move your hungry black hole, swallow buildings, cars, and even opponents to grow bigger before time runs out. The more you absorb, the stronger you become. Can you outsize the competition and take over the arena?\n\nKey Features:\n- Addictive black hole gameplay – Swallow objects and expand\n- Real-time multiplayer battles – Compete against other players\n- Time-based challenges – Grow fast before the clock runs out\n- Custom skins – Choose your favorite black hole design\n\nDownload Hole.io now and prove you’re the ultimate hole master in this fast-paced, city-eating battle!",
"icon": "https://play-lh.googleusercontent.com/VmbS-Smui9i1h9mxAVxnehbLI4HP4g-wsdWeJ2k9MbOTFK5wsiZifDkxdkm4f4wEIsE",
"screenshots": [
"https://play-lh.googleusercontent.com/l17Dh8NbbMAy9IjqLaWLgpexLyUtgcRTCjLC9lIBSS6KZyJAJc8t1L0IC-pdkp_EPMA",
"https://play-lh.googleusercontent.com/cHZjod0BGZyzPmxU47x1kTV5qjTkiOxHnSnU_mh8JA84Gu1JlfaGrjEPRNeHfX_NZg",
"https://play-lh.googleusercontent.com/ASB65zbzKNq4E1qbaddMVd-8p6yRjBisTONCtH9Ezdp4h9VQLHS2J7Tz5ifcasQFy4Zl",
"https://play-lh.googleusercontent.com/m34rd3EHiVhKljvhzvZEOn0TNO0gn1gssj0cNSj8BAMTtlv4c0g2EXTkjEkaLGYLni4",
"https://play-lh.googleusercontent.com/U8X6FE9YH2auqG7gmiOLnhVdWHM1aBBRHKPk-YhJyl30605oyF74icpBrYnD5CdrKzbN",
"https://play-lh.googleusercontent.com/ePjsmRoAypHloOSmlMDheHD-yRwJK0gDuyYXytKtkDMirxXdxnmQzwr7_zsfir2IIA",
"https://play-lh.googleusercontent.com/WSgXQ93MCO-8hXSz4N2Wlnj7wARPaGbpqbIpw0Pd3SldIOETo6cZvznQ-GY76fh0o-Q",
"https://play-lh.googleusercontent.com/Pyxg9HsuKHmqc0toUUXXNfv9drxAvnYEjd5ZMFJtjPSYd8uyMUv6jXe6PCh1Vr0KR3ww",
"https://play-lh.googleusercontent.com/-zYzl17Q2h8ToJIrYkaXC499PGEETGgwPUzpjAUeTM2plVXvac_4SX8NfUKSNg9L4g",
"https://play-lh.googleusercontent.com/wiEF1waTb8W1hecf7aCwCubP7mlLdCJkhXlXRlYJ8-rPZdABigdTARRqgOX2kgM2UA",
"https://play-lh.googleusercontent.com/BmONoE5HRs3UofP4HGfotSYIuZXch1RIHM3qr_Z-sNZmRGyIj_R1IyNnEZ6fJFKuEA",
"https://play-lh.googleusercontent.com/5KZ4ssKf6g8BfwCig8sw7IxHRAXQdqOIBbgZSTcoZmh3950CDgx1YIAPzX0JN3EDW88",
"https://play-lh.googleusercontent.com/gCzvKOhE7IEvitfSqn81M7Cevw80UzQtxIwkkoKzsXW5aL9DnB1yqKMD5Oo4unRURFUc",
"https://play-lh.googleusercontent.com/n6ec3XKolKYFtimDGyh8u_jIw5aWyn7W9ieb8nARdJZnEs8Od_D5LTd-Mz-yA5fSVA",
"https://play-lh.googleusercontent.com/J5e8ofyy9hpon5o5d_bw-ZZN62ITjdCdeVuY7hpV8cGr9tOZPB8Htimcaw702tuOMPE",
"https://play-lh.googleusercontent.com/EYl0uHi8GtLsNAqSpXiheOJ-5wwLOV7E1lE3oXZaak4qsNUoigYXaHQA1Npz6HEkMyk",
"https://play-lh.googleusercontent.com/w7frGmcLkDK-VhqKFNhEqbrj-6nAmhN5u3cz74uE-CzwZoSjrEuTH5WwX01wyq0ElFQ",
"https://play-lh.googleusercontent.com/M_1W90HoWOFkWUG5sLsgzJX0u1hBwptimTnVNdv1IQwnP0Dn7MK8tDdqbexZbJeV_0Q",
"https://play-lh.googleusercontent.com/_DJsKJotpIs43RudiUu9a6lq1tLUKcOYYQUlt362b_0MbXMI984CJEMkngC2g3-A47Pa",
"https://play-lh.googleusercontent.com/PFpYV_GNhunDKtkg55HUT7FtT_59g57oFdePKFfcOT1L9nVADmEK13iX7B1LlqeB5cA",
"https://play-lh.googleusercontent.com/Ib4lIp0U7K7lWrAF2M52JFBDcqwmMvTVeZnzo9j0KoyG8fQDuaikvTlN-FP4XTLb1suq",
"https://play-lh.googleusercontent.com/HALmyMfC0TKhLo4aYQkN8VJR4jw5nJIrAdDDkhTaky7SrwaBFS3XKB6AqTMuEI7v3Ug",
"https://play-lh.googleusercontent.com/b9gZMcPQkeRALpXg42Z5T14ASA3lpk8lwORYczdDzCwW6C_03JKwbsZI2D-IuVDWpqWa",
"https://play-lh.googleusercontent.com/AkkyxFJRMkGwyIdX7qYhCeOsbrsO6ct7j9zQ3DJvfAFHBEUR_l0lPxCmYcg_1wPYu0U"
],
"developer": "VOODOO",
"genre": "Arcade",
"score": 3.0541568,
"scoreText": "3.1",
"installs": "100,000,000+",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=io.voodoo.holeio"
}
]
================================================
FILE: output/reviews_example.json
================================================
[
{
"reviewId": "89d0170a-6e0a-4fd3-8f9c-b52ed975b701",
"userName": "Athul L Kumar",
"userImage": "https://play-lh.googleusercontent.com/a/ACg8ocIht0HMzD3U5L-Ck8nnt4caVTACxTkF4w8clfPx1hrFQQOwXw=mo",
"score": 1,
"content": "chapter 35 got a bug",
"thumbsUpCount": 0,
"appVersion": "1.21",
"at": "2025-10-13T10:42:05"
},
{
"reviewId": "ae919cd4-aee9-432e-833f-d63201dff153",
"userName": "Gourab Bhowal",
"userImage": "https://play-lh.googleusercontent.com/a-/ALV-UjVFglo1B-E1qE58LCO75T_YVZft2WsWbicrY2ClfJzY1U0JSxsX",
"score": 5,
"content": "Hands down one of the best games on the app...",
"thumbsUpCount": 0,
"appVersion": "1.21",
"at": "2025-10-12T09:37:21"
},
{
"reviewId": "839ae108-0edc-49ae-9f1b-6d5885a2705d",
"userName": "Ryl Ino",
"userImage": "https://play-lh.googleusercontent.com/a/ACg8ocLeqm0FMKCuVXygngB5zk3hu05YAbAzfYhyxPSejLtgWXZ0=mo",
"score": 5,
"content": "I enjoyed this game a lot and the cliff hanger ending was beautiful yet makes you want for more. ❤️",
"thumbsUpCount": 0,
"appVersion": null,
"at": "2025-10-12T08:01:08"
},
{
"reviewId": "ad0505b9-2c37-44d7-9ac8-dce40c125d96",
"userName": "Diederik Greef",
"userImage": "https://play-lh.googleusercontent.com/a-/ALV-UjWk0CkHOqbZSiznb4pSaA-RqLlTtF9nAU11vP2UpkCttYk41JU",
"score": 4,
"content": "very nice game I like it a lot but I just need more levels because I completed all of those levels with in a very small time limit is perfect please add more levels. game play experience some of the level's was looking like if it's impossible to play it but it's actually eazy when you see the puzzle in front of you but for sure need more levels it's not enough to keep someone playing. thank you it's more of a brain testing this game but it's good I recommend. but need more levels please.",
"thumbsUpCount": 0,
"appVersion": "1.21",
"at": "2025-10-12T02:29:57"
},
{
"reviewId": "007acb04-ffb8-4c38-974b-b8986509405f",
"userName": "Thinker Goat",
"userImage": "https://play-lh.googleusercontent.com/a-/ALV-UjVrMycBA0ee8fZbohoX2ifOssMp8edFwril5AQY3iAadHcBc9W6xA",
"score": 5,
"content": "This game is art",
"thumbsUpCount": 0,
"appVersion": "1.21",
"at": "2025-10-11T19:40:04"
},
{
"reviewId": "83f849df-f5a8-4df1-a0d2-2a83d5010d9a",
"userName": "Dinu Thomas",
"userImage": "https://play-lh.googleusercontent.com/a-/ALV-UjUFmqkINGFA75ErquMDMbvNZWyle6U-c-XU82Q9XUQH8fJ-fkjO",
"score": 5,
"content": "amazing game... i like it... difficult puzzles nice 💗",
"thumbsUpCount": 0,
"appVersion": "1.21",
"at": "2025-10-11T08:47:59"
},
{
"reviewId": "1120eabb-c7d5-4473-9693-4f950bf79fc0",
"userName": "Victor",
"userImage": "https://play-lh.googleusercontent.com/a-/ALV-UjWfgWOZ2uY5sF8jCIguoWYmL6-1TEe8emfpfb-Pbvx0dbBhQkdh",
"score": 5,
"content": "Definitely irritating to say the least, but amazing",
"thumbsUpCount": 0,
"appVersion": "1.21",
"at": "2025-10-11T08:47:49"
},
{
"reviewId": "6885b0da-07ff-474b-bca6-5e1abaeceefd",
"userName": "Ђорђе Вучковић",
"userImage": "https://play-lh.googleusercontent.com/a-/ALV-UjV8TBgfQqx-QYiRlntJ5L_PWO0P2ktdSLJKPvUpzM7ONa4uIP92Fg",
"score": 5,
"content": "A masterpiece.",
"thumbsUpCount": 0,
"appVersion": "1.21",
"at": "2025-10-10T19:25:33"
},
{
"reviewId": "de6ff595-c0c3-48b2-a2b6-83468ffd534f",
"userName": "Rajat Koparde",
"userImage": "https://play-lh.googleusercontent.com/a-/ALV-UjV1Q6yEdZyiaXX2NrQpgQJVTFRqjeUiNwWnTn7wvPi8iG6kSMmU",
"score": 5,
"content": "This game is very well made, I played it like 5-6 years ago, playing it now feels so nostalgic",
"thumbsUpCount": 0,
"appVersion": "1.21",
"at": "2025-10-09T19:59:39"
},
{
"reviewId": "d2d37b8a-ac73-4450-b78b-ea2ad7b39fcf",
"userName": "Retshepile Phori",
"userImage": "https://play-lh.googleusercontent.com/a-/ALV-UjVCgeoENmL3hLpr0AWTa1_jdA6vXqS8KO8ND1-6Qra4PEDtvek",
"score": 5,
"content": "motsamai",
"thumbsUpCount": 0,
"appVersion": null,
"at": "2025-10-09T08:16:21"
}
]
================================================
FILE: output/search_example.json
================================================
[
{
"appId": "com.instagram.android",
"title": "Instagram",
"description": "Create & share photos, stories, & reels with friends you love",
"icon": "https://play-lh.googleusercontent.com/VRMWkE5p3CkWhJs6nv-9ZsLAs1QOg5ob1_3qg-rckwYW7yp1fMrYZqnEFpk0IoVP4LM",
"developer": "Instagram",
"score": 4.4554186,
"scoreText": "4.5",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.instagram.android"
},
{
"appId": "com.twitter.android",
"title": "X",
"description": "Breaking News & Social Media",
"icon": "https://play-lh.googleusercontent.com/A-Rnrh0J7iKmABskTonqFAANRLGTGUg_nuE4PEMYwJavL3nPt5uWsU2WO_DSgV_mOOM",
"developer": "X Corp.",
"score": 4.052508,
"scoreText": "4.1",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.twitter.android"
},
{
"appId": "com.snapchat.android",
"title": "Snapchat",
"description": "Share the moment!",
"icon": "https://play-lh.googleusercontent.com/KxeSAjPTKliCErbivNiXrd6cTwfbqUJcbSRPe_IBVK_YmwckfMRS1VIHz-5cgT09yMo",
"developer": "Snap Inc",
"score": 3.9922094,
"scoreText": "4.0",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.snapchat.android"
},
{
"appId": "com.facebook.katana",
"title": "Facebook",
"description": "Explore the things you love",
"icon": "https://play-lh.googleusercontent.com/KCMTYuiTrKom4Vyf0G4foetVOwhKWzNbHWumV73IXexAIy5TTgZipL52WTt8ICL-oIo",
"developer": "Meta Platforms, Inc.",
"score": 4.4118733,
"scoreText": "4.4",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.facebook.katana"
},
{
"appId": "com.zhiliaoapp.musically",
"title": "TikTok",
"description": "Videos, Music & Live Streams",
"icon": "https://play-lh.googleusercontent.com/waX_CXnriskbccUeOevisOQwwR-tlPRPX0hiFZ4X-w2nHDTb_I2yeBliFX5VAqfetw",
"developer": "TikTok Pte. Ltd.",
"score": 4.2110004,
"scoreText": "4.2",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.zhiliaoapp.musically"
},
{
"appId": "com.pinterest",
"title": "Pinterest",
"description": "One destination for a world of inspiration.",
"icon": "https://play-lh.googleusercontent.com/6CFQQ0b9r5fzF1v6f0gIirWsOGL7sGWkJifuUQxxhbCMcBx5aSG_cNXpjDKDn5c1jwjq",
"developer": "Pinterest",
"score": 4.6440973,
"scoreText": "4.6",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.pinterest"
},
{
"appId": "com.tumblr",
"title": "Tumblr - Social Media Fandom",
"description": "Dive into diverse fandoms. Connect, create, reblog.",
"icon": "https://play-lh.googleusercontent.com/G4R3eZm3sDGJk0lVCOr72BwLFIYV0Jg5G7_PBOf9ZMpWwZjXaUdoZMyjFbJwxcmF5qBH",
"developer": "Tumblr, Inc",
"score": 4.4767933,
"scoreText": "4.5",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.tumblr"
},
{
"appId": "com.reddit.frontpage",
"title": "Reddit",
"description": "Find your community. Forums, threads, debates. Not just social networking fluff.",
"icon": "https://play-lh.googleusercontent.com/NaFAbO7ExS4NRAvt2GYkNY6OQf9oVXwmdMTZzA6zrgjjSxhQuTCnjHyf7TgYcoSGqQ",
"developer": "reddit Inc.",
"score": 4.6419964,
"scoreText": "4.6",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.reddit.frontpage"
},
{
"appId": "com.instagram.barcelona",
"title": "Threads",
"description": "Connect and share ideas",
"icon": "https://play-lh.googleusercontent.com/G6jK9S77RN0laf9_6nhDo3AVxbRP9SgMmt8ZmQjKQ2hibn9xhOY-W5YFn_7stJD1CA",
"developer": "Instagram",
"score": 4.322976,
"scoreText": "4.3",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.instagram.barcelona"
},
{
"appId": "com.discord",
"title": "Discord - Talk, Play, Hang Out",
"description": "Group Chat That’s Fun & Games",
"icon": "https://play-lh.googleusercontent.com/0oO5sAneb9lJP6l8c6DH4aj6f85qNpplQVHmPmbbBxAukDnlO7DarDW0b-kEIHa8SQ",
"developer": "Discord Inc.",
"score": 4.3991246,
"scoreText": "4.4",
"currency": null,
"price": 0,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.discord"
}
]
================================================
FILE: output/similar_example.json
================================================
[
{
"appId": "com.playdigious.littlenightmare",
"title": "Little Nightmares",
"description": "First available on PC and consoles, the horror adventure tale Little Nightmares is available on mobile!\nImmerse yourself in Little Nightmares, a dark whimsical tale that will confront you with your childhood fears!\nHelp Six escape The Maw – a vast, mysterious vessel inhabited by corrupted souls looking for their next meal.\nAs you progress on your journey, explore the most disturbing dollhouse offering a prison to escape from and a playground full of secrets to discover.\nReconnect with your inner child to unleash your imagination and find the way out!\nLittle Nightmares features a subtle mix of action and puzzle-platformer mechanics rooted in an eerie artistic direction and creepy sound design.\nSneak your way out of the Maw’s dreary maze and run from its corrupted inhabitants to escape your childhood fears. \n\nFEATURES\n\n- Tiptoe your way through a dark and thrilling adventure\n- Rediscover your childhood fears inside a haunting vessel and escape its eerie inhabitants\n- Climb, crawl and hide through nightmarish environments to solve tricky platform puzzles\n- Immerse yourself in the Maw through its creepy sound design\n\nPlease make sure your device is connected to Wifi to download the game for the first time.\n\nIf you run into a problem, please contact us at https://playdigious.helpshift.com/hc/en/12-playdigious/ with as much information as possible on the issue.",
"icon": "https://play-lh.googleusercontent.com/vEJAwZwhOv7Wzf1Md7PsMzOyo087y0Z4rhRgidtfv03c682RicoBz3BOsrigdiXA-7I",
"developer": "Playdigious",
"score": 4.27,
"scoreText": "4.3",
"currency": "USD",
"price": 8.99,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.playdigious.littlenightmare"
},
{
"appId": "com.agaming.reporter",
"title": "Reporter - Scary Horror Game",
"description": "Discover heart-thumping terror, chase and scary creatures in this atmospheric horror game. But no matter what, never play alone in the dark.\n\nIs Your mind is hungry for tricky puzzles and nerves are suffering without ticklish situations? This action-horror \"Reporter\" from \"AGaming+\" will shake you to the core! Turn off the light and get your earphones! Be attentive, because it’s the only thing that can help you to get out from the paws of horror that is happening here. \n\nAll the story started in a small town. Once on a wonderful day, the town was shocked by series of horrible kills under terrifying and inexplicable circumstances. Police have tried to hide the facts, but some information leaked and was published by local press. Trying to understand what has happened there, you start your search for the truth. But suddenly, you become the part of the story that you'll remember till the end of your life. It all depends on you, will you unravel this plexus of horror and chaos and survive?\n\nWarning: for perfect experience and correct game functioning 1GB of RAM is required.",
"icon": "https://play-lh.googleusercontent.com/ApOs3N1Qgf4bZGpaBI4DK7DKokTa-db309_RHCoSGuY2cKq6JpX59Sg1C2WhJDKirw",
"developer": "AGaming+",
"score": 3.61,
"scoreText": "3.6",
"currency": "USD",
"price": 0.99,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.agaming.reporter"
},
{
"appId": "com.cowcat.brok",
"title": "BROK the InvestiGator",
"description": "BROK is an innovative adventure mixed with beat 'em up and RPG elements. In a grim world where animals have replaced mankind, what kind of detective will you be?\n\nIn a futuristic \"light cyberpunk\" world where animals have replaced humans, privileged citizens live under a protective dome from the ambient air pollution while others struggle to make a living on the outside.\n\nBrok, a private detective and former boxer, lives with Graff, the son of his deceased wife. Although he could never elucidate her accident, recent events may shed some light on an even more tragic outcome... one that may be linked to their own existence. \nWill they be able to withstand the threats of this corrupted world and face their own destiny? \n\n--------------------\nFEATURES\n--------------------\n- Solve puzzles with your wits... or muscles!\n- Make choices impacting gameplay and/or story\n- Relaxed mode for pure \"Point & Click\" gameplay (fights can be skipped)\n- Level up to beat enemies and bosses\n- Combine clues to uncover the truth!\n- In-game hints\n- Two playable characters, switch at any time\n- 15 to 20 hours long on first playthrough \n- Multiple distinct endings to unlock \n- Fully voice acted (23,000 lines)\n- Optimized for touch screens (fight using touch swipes or virtual buttons)\n- Compatible with most Bluetooth controllers\n- Play the adventure with friends in local co-op (up to 4 players)\n- Text fully translated into 10 languages\n\n---------------------------------\nACCESSIBILITY\n---------------------------------\nBROK is the first full-fledged adventure game to be fully playable by blind or visually impaired players!\n\n- Fully narrated via quality text-to-speech and audiodescriptions (characters, locations and scenes.)\n- Puzzles adapted for blindness.\n- All puzzles and fights can be skipped.\n- Adapted tutorials.\n- Ability to repeat the last voice speech and instructions.\n- Positional audio for fights.\n- No online connectivity required (after the download).\n- No specific device required\n- Additional options: larger fonts and increased contrast (backgrounds and enemies.)\n\nTo enter the accessibility menu, press two fingers on the title screen, then follow the audio instructions.\n\nIMPORTANT: Accessibility speeches are only available in English.\n\n---------------------------------\nMONETIZATION\n---------------------------------\n- Chapter 1 is entirely free (2 to 3 hours of gameplay)\n- Each additional chapter is $1.99\n- An alternative Premium option to buy all chapters at once is $7.99 (the game has 6 chapters)",
"icon": "https://play-lh.googleusercontent.com/XQFeA6xQ9TgaQmKYI-OSkEOPlZpnvCdHUxT-x2GqagCbDkIByAl2V2JgcXx9-ZnEoA",
"developer": "Breton Fabrice",
"score": 4.65,
"scoreText": "4.7",
"currency": "USD",
"price": 0,
"free": true,
"url": "https://play.google.com/store/apps/details?id=com.cowcat.brok"
},
{
"appId": "com.MorionStudio.ConquistadorioFull",
"title": "Conquistadorio",
"description": "Embark on an enthralling journey with Conquistodoro, a captivating point-and-click adventure that beckons you to unravel mysteries, solve puzzles, and conquer challenging trials. In this immersive world, every click propels you deeper into the heart of the adventure, where the thrill of exploration meets the satisfaction of clever point-and-click interactions.\n\n🗺️ Discover a World of Adventure\n\nSet in a mysterious world, Conquistodoro promises a rich narrative filled with twists and turns. Your protagonist, a daring bandit, faces expulsion from his coffin and sets forth on an epic quest to find a new resting place. Navigate through the story, guided by helpful souls and driven by the need to secure a mysterious cup that holds the key to reviving zombies in ancient tombs.\n\n🧩 Point-and-Click Challenges Await\n\nAs you progress, encounter a variety of challenges and puzzles that will test your point-and-click skills. Each decision you make shapes the outcome, adding layers of complexity to the adventure. Conquistodoro ensures that your journey is not just a visual feast but a mental challenge, with every click contributing to the unfolding saga.\n\n🤠 The Hero's Quest Unfolds\n\nJoin forces with a charismatic protagonist as he faces adversity, completes missions, and navigates through a visually stunning world. The commander, zombies, and a boss beetle become integral parts of this thrilling adventure, where every point-and-click action propels the hero closer to his ultimate goal.\n\n🎨 Stunning Visuals and Animations\n\nImmerse yourself in Conquistodoro's visually captivating world, where stunning graphics and animations bring the narrative to life. The meticulously crafted design enhances the overall point-and-click experience, ensuring that every interaction is a visual treat.\n\n🎶 Adventure with an Enchanting Score\n\nAccompanying your quest is an enchanting musical score that heightens the atmosphere and emotion of the game. The soundtrack is thoughtfully curated to elevate the overall adventure, making Conquistodoro a symphony of point-and-click excitement and immersive storytelling.\n\n📲 Download Now and Embark on Your Adventure!\n\nReady to experience the magic of point-and-click adventure? Download Conquistodoro now and dive into a world where each click takes you closer to solving puzzles, overcoming challenges, and unraveling the mysteries that await. Your adventure begins – click your way to triumph in Conquistodoro!",
"icon": "https://play-lh.googleusercontent.com/MC1Dspw44XFvGizJfpH90LjlCyuUHuPSxg37m2Wbq6dbzO4KsLcYDNAB8EGVge7L_yc",
"developer": "Morion Studio",
"score": 4.32,
"scoreText": "4.3",
"currency": "USD",
"price": 4.99,
"free": false,
"url": "https://play.google.com/store/apps/details?id=com.MorionStudio.ConquistadorioFull"
},
{
"appId": "eu.bandainamcoent.verylittlenightmares",
"title": "Very Little Nightmares",
"description": "Enter the world of Very Little Nightmares, a puzzle adventure game that mixes a cute and creepy universe. 👻\n\nHelp the Girl in the Yellow Raincoat survive in a hostile house and find a way to get her out. 💛\n\nAs she awakens in an unknown mansion, you must guide her through each room. What a fate to fall here, a place where everything wants to see her dead. 💀\n\nHer life is in your hands, avoid enemies, discover intriguing puzzles to finally pierce the secrets of this strange house. 🏚\n\n=====\n\nEXPLORE 🔎\nThe Nest, a vast maze filled with life-threatening traps.\n\nSOLVE 💡\nThe challenging puzzles that bar your way. Use your wits and any resources at your disposal.\n\nSURVIVE 😬\nThe frightening enemies that will do everything to capture you.\n\nDISCOVER 🕵♀\nA dark universe in this original prequel story of the events in Little Nightmares.\n\nKeep in touch & unveil the mysteries of this world:\n\nFacebook: https://www.facebook.com/LittleNightmaresEU/ \nTwitter: https://twitter.com/LittleNights \nInstagram: https://www.instagram.com/little__nightmares/ \n\nYou are purchasing a license for digital goods. For full terms and conditions, please see the License Agreement below.\n\n⭐ SUPPORT: Having problems? Let us know at http://bnent.eu/msupportvln\n⭐ PRIVACY POLICY: http://bnent.eu/mprivacy\n⭐ TERMS OF USE: http://bnent.eu/mterms",
"icon": "https://play-lh.googleusercontent.com/aSfffFeiMMNDux_Vet9VyQ97_Pt0XBJwZeCOVPBF6swycKN1sdc_gCDyuwXeR4CVvMc",
"developer": "BANDAI NAMCO Entertainment Europe",
"score": 4.4785895,
"scoreText": "4.5",
"currency": "USD",
"price": 6.99,
"free": false,
"url": "https://play.google.com/store/apps/details?id=eu.bandainamcoent.verylittlenightmares"
}
]
================================================
FILE: output/suggest_example.json
================================================
{
"photo editor": [
"photo editor",
"photo editor free",
"photo editor app",
"photo editor 2023",
"photo editor free app android"
],
"photo editor free": [
"photo editor free",
"photo editor free app android",
"photo editor free app",
"photo editor free app android free",
"photo editor free 2025"
],
"photo editor app": [
"photo editor app",
"photo editor app new style 2020",
"photo editor app free",
"photo editor app new style 2025",
"photo editor app 2025"
],
"photo editor 2023": [
"photo editor 2023",
"photo editor 2023 video song",
"photo editor 2023 background",
"photo editor 2023 new app",
"photo editor 2023 background change"
],
"photo editor free app android": [
"photo editor free app android",
"photo editor free app android offline",
"photo editor free app android free",
"photo editor free app android 2025"
]
}
================================================
FILE: requirements.txt
================================================
# Core dependencies
requests>=2.25.0
beautifulsoup4>=4.9.0
# HTTP client libraries
curl-cffi>=0.5.0
tls-client>=1.0.0
urllib3>=1.26.0
cloudscraper>=1.2.0
aiohttp>=3.8.0
httpx>=0.24.0
# Development dependencies
pytest>=6.0.0
# Documentation dependencies
sphinx>=4.0.0
sphinx-rtd-theme>=1.0.0
================================================
FILE: setup.py
================================================
from setuptools import setup, find_packages
with open("README.md", "r", encoding="utf-8") as fh:
long_description = fh.read()
with open("requirements.txt", "r", encoding="utf-8") as fh:
requirements = [line.strip() for line in fh if line.strip() and not line.startswith("#")]
setup(
name="gplay-scraper",
version="1.0.6",
description="🚀 Advanced Google Play Store Scraper - Extract 65+ app fields, reviews, ratings, ASO data, developer info, top charts, search results with 7 HTTP clients & unlimited pagination support",
long_description=long_description,
long_description_content_type="text/markdown",
author="Mohammed Cha",
author_email="contact@mohammedcha.com",
maintainer="Mohammed Cha",
maintainer_email="contact@mohammedcha.com",
url="https://github.com/mohammedcha/gplay-scraper",
download_url="https://github.com/mohammedcha/gplay-scraper/archive/v1.0.6.tar.gz",
project_urls={
"Homepage": "https://github.com/mohammedcha/gplay-scraper",
"Documentation": "https://mohammedcha.github.io/gplay-scraper/",
"Source Code": "https://github.com/mohammedcha/gplay-scraper",
"Bug Reports": "https://github.com/mohammedcha/gplay-scraper/issues",
"Feature Requests": "https://github.com/mohammedcha/gplay-scraper/issues",
"Changelog": "https://github.com/mohammedcha/gplay-scraper/blob/main/CHANGELOG.md",
"Examples": "https://github.com/mohammedcha/gplay-scraper/tree/main/examples",
"PyPI": "https://pypi.org/project/gplay-scraper/",
},
packages=find_packages(exclude=["tests*", "docs*", "examples*"]),
install_requires=requirements,
extras_require={
"dev": ["pytest>=7.0.0", "pytest-cov>=4.0.0", "black>=22.0.0", "flake8>=5.0.0"],
"http-clients": [
"curl-cffi>=0.5.0",
"tls-client>=0.2.0",
"httpx>=0.24.0",
"urllib3>=1.26.0",
"cloudscraper>=1.2.0",
"aiohttp>=3.8.0",
],
"all": [
"pytest>=7.0.0", "pytest-cov>=4.0.0", "black>=22.0.0", "flake8>=5.0.0",
"curl-cffi>=0.5.0", "tls-client>=0.2.0", "httpx>=0.24.0",
"urllib3>=1.26.0", "cloudscraper>=1.2.0", "aiohttp>=3.8.0",
],
},
python_requires=">=3.8",
keywords="google-play-scraper, playstore-scraper, android-scraper, gplay-scraper, google-play-store, play-store-api, app-data-extraction, app-analytics, mobile-analytics, aso-tools, app-store-optimization, mobile-seo, app-marketing, keyword-research, competitor-analysis, market-research, app-reviews, user-reviews, review-scraper, rating-analysis, sentiment-analysis, developer-tools, api-scraping, web-scraping, data-mining, python-scraper, automation-tools, business-intelligence, market-intelligence, competitive-intelligence, app-monitoring, trend-analysis, performance-tracking, install-tracking, revenue-analysis",
classifiers=[
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Intended Audience :: Information Technology",
"Intended Audience :: Science/Research",
"Intended Audience :: Financial and Insurance Industry",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Operating System :: Microsoft :: Windows",
"Operating System :: POSIX :: Linux",
"Operating System :: MacOS",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3 :: Only",
"Topic :: Internet :: WWW/HTTP :: Dynamic Content",
"Topic :: Internet :: WWW/HTTP :: Indexing/Search",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: Software Development :: Libraries :: Application Frameworks",
"Topic :: Utilities",
"Topic :: Scientific/Engineering :: Information Analysis",
"Topic :: Office/Business :: Financial :: Investment",
"Topic :: Internet :: WWW/HTTP :: Browsers",
"Topic :: Text Processing :: Markup :: HTML",
"Topic :: Database",
"Natural Language :: English",
"Environment :: Console",
"Environment :: Web Environment",
"Framework :: AsyncIO",
"Typing :: Typed",
],
platforms=["any"],
include_package_data=True,
zip_safe=False,
)
================================================
FILE: tests/__init__.py
================================================
# Tests package
================================================
FILE: tests/test_app_methods.py
================================================
import unittest
import warnings
import time
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import GPlayScraperError, NetworkError, RateLimitError
class TestAppMethods(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.scraper = GPlayScraper() # Initialize scraper
cls.app_id = "com.whatsapp" # WhatsApp app ID for testing
cls.lang = "en" # Language
cls.country = "us" # Country
def test_app_analyze(self):
"""Test app_analyze returns dictionary with data or handles errors gracefully"""
time.sleep(2) # Wait 2 seconds before request
try:
result = self.scraper.app_analyze(self.app_id, lang=self.lang, country=self.country)
self.assertIsInstance(result, dict)
if result: # Only check if we got data
self.assertIn('title', result)
print(f"\n✅ App data retrieved for {self.app_id}:")
print(f"Title: {result.get('title', 'N/A')}")
print(f"Score: {result.get('score', 'N/A')}")
print(f"Installs: {result.get('installs', 'N/A')}")
print(f"Developer: {result.get('developer', 'N/A')}")
print(f"Total fields: {len(result)}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_app_analyze: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_app_get_field(self):
"""Test app_get_field returns single field value or handles errors gracefully"""
time.sleep(2) # Wait 2 seconds before request
try:
result = self.scraper.app_get_field(self.app_id, "title", lang=self.lang, country=self.country)
if result is not None:
self.assertIsInstance(result, str)
self.assertTrue(len(result) > 0)
print(f"\n✅ Single field 'title': {result}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_app_get_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_app_get_fields(self):
"""Test app_get_fields returns multiple fields or handles errors gracefully"""
time.sleep(2) # Wait 2 seconds before request
fields = ["title", "score", "installs"]
try:
result = self.scraper.app_get_fields(self.app_id, fields, lang=self.lang, country=self.country)
if result:
self.assertIsInstance(result, dict)
print(f"\n✅ Multiple fields retrieved:")
for field, value in result.items():
print(f" {field}: {value}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_app_get_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_app_print_field(self):
"""Test app_print_field executes without error or handles errors gracefully"""
time.sleep(2) # Wait 2 seconds before request
try:
print(f"\n✅ app_print_field output:")
self.scraper.app_print_field(self.app_id, "title", lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_app_print_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"app_print_field raised unexpected {type(e).__name__}: {e}")
def test_app_print_fields(self):
"""Test app_print_fields executes without error or handles errors gracefully"""
time.sleep(2) # Wait 2 seconds before request
try:
print(f"\n✅ app_print_fields output:")
self.scraper.app_print_fields(self.app_id, ["title", "score"], lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_app_print_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"app_print_fields raised unexpected {type(e).__name__}: {e}")
def test_app_print_all(self):
"""Test app_print_all executes without error or handles errors gracefully"""
time.sleep(2) # Wait 2 seconds before request
try:
print(f"\n✅ app_print_all output:")
self.scraper.app_print_all(self.app_id, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_app_print_all: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"app_print_all raised unexpected {type(e).__name__}: {e}")
if __name__ == '__main__':
unittest.main()
================================================
FILE: tests/test_basic.py
================================================
import unittest
import sys
import os
# Add the parent directory to the path to import gplay_scraper
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from gplay_scraper import GPlayScraper
class TestBasicFunctionality(unittest.TestCase):
"""Basic tests that don't require network access"""
def test_import_gplay_scraper(self):
"""Test that GPlayScraper can be imported"""
from gplay_scraper import GPlayScraper
self.assertTrue(GPlayScraper is not None)
def test_scraper_initialization(self):
"""Test that GPlayScraper can be initialized"""
scraper = GPlayScraper()
self.assertIsInstance(scraper, GPlayScraper)
def test_scraper_initialization_with_http_client(self):
"""Test that GPlayScraper can be initialized with different HTTP clients"""
# Test with requests (default)
scraper1 = GPlayScraper(http_client="requests")
self.assertIsInstance(scraper1, GPlayScraper)
# Test with curl_cffi
try:
scraper2 = GPlayScraper(http_client="curl_cffi")
self.assertIsInstance(scraper2, GPlayScraper)
except ImportError:
# curl_cffi might not be installed in CI
pass
def test_scraper_has_required_methods(self):
"""Test that GPlayScraper has all required methods"""
scraper = GPlayScraper()
# App methods
self.assertTrue(hasattr(scraper, 'app_analyze'))
self.assertTrue(hasattr(scraper, 'app_get_field'))
self.assertTrue(hasattr(scraper, 'app_get_fields'))
self.assertTrue(hasattr(scraper, 'app_print_field'))
self.assertTrue(hasattr(scraper, 'app_print_fields'))
self.assertTrue(hasattr(scraper, 'app_print_all'))
# Search methods
self.assertTrue(hasattr(scraper, 'search_analyze'))
self.assertTrue(hasattr(scraper, 'search_get_field'))
self.assertTrue(hasattr(scraper, 'search_get_fields'))
self.assertTrue(hasattr(scraper, 'search_print_field'))
self.assertTrue(hasattr(scraper, 'search_print_fields'))
self.assertTrue(hasattr(scraper, 'search_print_all'))
# Reviews methods
self.assertTrue(hasattr(scraper, 'reviews_analyze'))
self.assertTrue(hasattr(scraper, 'reviews_get_field'))
self.assertTrue(hasattr(scraper, 'reviews_get_fields'))
self.assertTrue(hasattr(scraper, 'reviews_print_field'))
self.assertTrue(hasattr(scraper, 'reviews_print_fields'))
self.assertTrue(hasattr(scraper, 'reviews_print_all'))
# Developer methods
self.assertTrue(hasattr(scraper, 'developer_analyze'))
self.assertTrue(hasattr(scraper, 'developer_get_field'))
self.assertTrue(hasattr(scraper, 'developer_get_fields'))
self.assertTrue(hasattr(scraper, 'developer_print_field'))
self.assertTrue(hasattr(scraper, 'developer_print_fields'))
self.assertTrue(hasattr(scraper, 'developer_print_all'))
# List methods
self.assertTrue(hasattr(scraper, 'list_analyze'))
self.assertTrue(hasattr(scraper, 'list_get_field'))
self.assertTrue(hasattr(scraper, 'list_get_fields'))
self.assertTrue(hasattr(scraper, 'list_print_field'))
self.assertTrue(hasattr(scraper, 'list_print_fields'))
self.assertTrue(hasattr(scraper, 'list_print_all'))
# Similar methods
self.assertTrue(hasattr(scraper, 'similar_analyze'))
self.assertTrue(hasattr(scraper, 'similar_get_field'))
self.assertTrue(hasattr(scraper, 'similar_get_fields'))
self.assertTrue(hasattr(scraper, 'similar_print_field'))
self.assertTrue(hasattr(scraper, 'similar_print_fields'))
self.assertTrue(hasattr(scraper, 'similar_print_all'))
# Suggest methods (only 4 methods, not 6)
self.assertTrue(hasattr(scraper, 'suggest_analyze'))
self.assertTrue(hasattr(scraper, 'suggest_nested'))
self.assertTrue(hasattr(scraper, 'suggest_print_all'))
self.assertTrue(hasattr(scraper, 'suggest_print_nested'))
if __name__ == '__main__':
unittest.main()
================================================
FILE: tests/test_developer_methods.py
================================================
"""
Unit tests for Developer Methods
"""
import unittest
import time
import warnings
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import GPlayScraperError, NetworkError, RateLimitError
class TestDeveloperMethods(unittest.TestCase):
"""Test suite for developer methods."""
def setUp(self):
"""Set up test fixtures before each test method."""
self.scraper = GPlayScraper() # Initialize scraper
self.dev_id = "5700313618786177705" # Google Inc. developer ID
self.count = 10 # Number of items to fetch
self.lang = "en" # Language
self.country = "us" # Country
def test_developer_analyze(self):
"""Test developer_analyze returns list of apps."""
time.sleep(2)
try:
result = self.scraper.developer_analyze(self.dev_id, count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
self.assertGreater(len(result), 0)
print(f"\n✅ Developer apps ({len(result)} apps):")
for i, app in enumerate(result[:3]): # Show first 3 apps
print(f" {i+1}. {app.get('title', 'N/A')} - {app.get('score', 'N/A')} stars")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_developer_analyze: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_developer_get_field(self):
"""Test developer_get_field returns list of field values."""
time.sleep(2)
try:
result = self.scraper.developer_get_field(self.dev_id, "title", count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
print(f"\n✅ Developer app titles ({len(result)} apps):")
for i, title in enumerate(result[:3]):
print(f" {i+1}. {title}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_developer_get_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_developer_get_fields(self):
"""Test developer_get_fields returns list of dictionaries."""
time.sleep(2)
try:
result = self.scraper.developer_get_fields(self.dev_id, ["title", "score"], count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
print(f"\n✅ Developer app fields ({len(result)} apps):")
for i, app in enumerate(result[:3]):
print(f" {i+1}. {app.get('title', 'N/A')} - {app.get('score', 'N/A')} stars")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_developer_get_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_developer_print_field(self):
"""Test developer_print_field executes without error."""
time.sleep(2)
try:
print(f"\n✅ developer_print_field output:")
self.scraper.developer_print_field(self.dev_id, "title", count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_developer_print_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"developer_print_field raised unexpected {e}")
def test_developer_print_fields(self):
"""Test developer_print_fields executes without error."""
time.sleep(2)
try:
print(f"\n✅ developer_print_fields output:")
self.scraper.developer_print_fields(self.dev_id, ["title", "score"], count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_developer_print_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"developer_print_fields raised unexpected {e}")
def test_developer_print_all(self):
"""Test developer_print_all executes without error."""
time.sleep(2)
try:
print(f"\n✅ developer_print_all output:")
self.scraper.developer_print_all(self.dev_id, count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_developer_print_all: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"developer_print_all raised unexpected {e}")
if __name__ == '__main__':
unittest.main()
================================================
FILE: tests/test_list_methods.py
================================================
"""
Unit tests for List Methods
"""
import unittest
import time
import warnings
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import GPlayScraperError, NetworkError, RateLimitError
class TestListMethods(unittest.TestCase):
"""Test suite for list methods (top charts)."""
def setUp(self):
"""Set up test fixtures before each test method."""
self.scraper = GPlayScraper() # Initialize scraper
self.collection = "TOP_FREE" # Top free apps collection
self.category = "GAME" # Game category
self.count = 10 # Number of items to fetch
self.lang = "en" # Language
self.country = "us" # Country
def test_list_analyze(self):
"""Test list_analyze returns list of top apps."""
time.sleep(2)
try:
result = self.scraper.list_analyze(self.collection, self.category, count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
self.assertGreater(len(result), 0)
print(f"\n✅ Top {self.collection} {self.category} apps ({len(result)} apps):")
for i, app in enumerate(result[:3]): # Show first 3 apps
print(f" {i+1}. {app.get('title', 'N/A')} - {app.get('developer', 'N/A')}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_list_analyze: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_list_get_field(self):
"""Test list_get_field returns list of field values."""
time.sleep(2)
try:
result = self.scraper.list_get_field(self.collection, self.category, "title", count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
print(f"\n✅ Top chart app titles ({len(result)} apps):")
for i, title in enumerate(result[:3]):
print(f" {i+1}. {title}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_list_get_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_list_get_fields(self):
"""Test list_get_fields returns list of dictionaries."""
time.sleep(2)
try:
result = self.scraper.list_get_fields(self.collection, self.category, ["title", "score"], count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
print(f"\n✅ Top chart app fields ({len(result)} apps):")
for i, app in enumerate(result[:3]):
print(f" {i+1}. {app.get('title', 'N/A')} - {app.get('score', 'N/A')} stars")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_list_get_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_list_print_field(self):
"""Test list_print_field executes without error."""
time.sleep(2)
try:
print(f"\n✅ list_print_field output:")
self.scraper.list_print_field(self.collection, self.category, "title", count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_list_print_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"list_print_field raised unexpected {e}")
def test_list_print_fields(self):
"""Test list_print_fields executes without error."""
time.sleep(2)
try:
print(f"\n✅ list_print_fields output:")
self.scraper.list_print_fields(self.collection, self.category, ["title", "score"], count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_list_print_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"list_print_fields raised unexpected {e}")
def test_list_print_all(self):
"""Test list_print_all executes without error."""
time.sleep(2)
try:
print(f"\n✅ list_print_all output:")
self.scraper.list_print_all(self.collection, self.category, count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_list_print_all: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"list_print_all raised unexpected {e}")
if __name__ == '__main__':
unittest.main()
================================================
FILE: tests/test_package.py
================================================
#!/usr/bin/env python3
"""
Simple test script to verify gplay-scraper package functionality
This script tests basic import and initialization without network calls
"""
import unittest
import sys
import os
# Add the parent directory to the path to import gplay_scraper
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from gplay_scraper import GPlayScraper
class TestPackageFunctionality(unittest.TestCase):
"""Package functionality tests that don't require network access"""
def test_import_gplay_scraper(self):
"""Test that GPlayScraper can be imported"""
from gplay_scraper import GPlayScraper
self.assertTrue(GPlayScraper is not None)
def test_scraper_initialization(self):
"""Test that GPlayScraper can be initialized"""
scraper = GPlayScraper()
self.assertIsInstance(scraper, GPlayScraper)
def test_http_clients(self):
"""Test different HTTP client initializations"""
clients = ["requests", "curl_cffi", "tls_client", "httpx", "urllib3", "cloudscraper", "aiohttp"]
success_count = 0
for client in clients:
try:
scraper = GPlayScraper(http_client=client)
self.assertIsInstance(scraper, GPlayScraper)
success_count += 1
except ImportError:
# Optional dependency not available
pass
self.assertGreater(success_count, 0, "At least one HTTP client should work")
def test_all_methods_exist(self):
"""Test that all expected methods exist"""
scraper = GPlayScraper()
method_groups = [
("app", ["analyze", "get_field", "get_fields", "print_field", "print_fields", "print_all"]),
("search", ["analyze", "get_field", "get_fields", "print_field", "print_fields", "print_all"]),
("reviews", ["analyze", "get_field", "get_fields", "print_field", "print_fields", "print_all"]),
("developer", ["analyze", "get_field", "get_fields", "print_field", "print_fields", "print_all"]),
("similar", ["analyze", "get_field", "get_fields", "print_field", "print_fields", "print_all"]),
("list", ["analyze", "get_field", "get_fields", "print_field", "print_fields", "print_all"]),
("suggest", ["analyze", "nested", "print_all", "print_nested"]),
]
total_methods = 0
for group, methods in method_groups:
for method in methods:
method_name = f"{group}_{method}"
if hasattr(scraper, method_name):
print(f"✓ Method {method_name} exists")
total_methods += 1
else:
print(f"✗ Method {method_name} missing")
self.fail(f"Method {method_name} missing")
print(f"\n✅ All {total_methods} methods found and working!")
self.assertEqual(total_methods, 40, "Should have exactly 40 methods")
if __name__ == '__main__':
unittest.main()
================================================
FILE: tests/test_reviews_methods.py
================================================
"""
Unit tests for Reviews Methods
"""
import unittest
import time
import warnings
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import GPlayScraperError, NetworkError, RateLimitError
class TestReviewsMethods(unittest.TestCase):
"""Test suite for reviews methods."""
def setUp(self):
"""Set up test fixtures before each test method."""
self.scraper = GPlayScraper() # Initialize scraper
self.app_id = "com.whatsapp" # WhatsApp app ID for testing
self.count = 10 # Number of items to fetch
self.lang = "en" # Language
self.country = "us" # Country
self.sort = "NEWEST" # Sort order for reviews
def test_reviews_analyze(self):
"""Test reviews_analyze returns list of reviews."""
time.sleep(2)
try:
result = self.scraper.reviews_analyze(self.app_id, count=self.count, sort=self.sort, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
self.assertGreater(len(result), 0)
print(f"\n✅ Reviews for {self.app_id} ({len(result)} reviews):")
for i, review in enumerate(result[:2]): # Show first 2 reviews
print(f" {i+1}. {review.get('userName', 'Anonymous')} - {review.get('score', 'N/A')} stars")
print(f" {review.get('content', 'No content')[:100]}...")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_reviews_analyze: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_reviews_get_field(self):
"""Test reviews_get_field returns list of field values."""
time.sleep(2)
try:
result = self.scraper.reviews_get_field(self.app_id, "userName", count=self.count, sort=self.sort, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
print(f"\n✅ Review usernames ({len(result)} reviews):")
for i, username in enumerate(result[:3]):
print(f" {i+1}. {username or 'Anonymous'}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_reviews_get_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_reviews_get_fields(self):
"""Test reviews_get_fields returns list of dictionaries."""
time.sleep(2)
try:
result = self.scraper.reviews_get_fields(self.app_id, ["userName", "score"], count=self.count, sort=self.sort, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
print(f"\n✅ Review fields ({len(result)} reviews):")
for i, review in enumerate(result[:3]):
print(f" {i+1}. {review.get('userName', 'Anonymous')} - {review.get('score', 'N/A')} stars")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_reviews_get_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_reviews_print_field(self):
"""Test reviews_print_field executes without error."""
time.sleep(2)
try:
print(f"\n✅ reviews_print_field output:")
self.scraper.reviews_print_field(self.app_id, "userName", count=self.count, sort=self.sort, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_reviews_print_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"reviews_print_field raised unexpected {e}")
def test_reviews_print_fields(self):
"""Test reviews_print_fields executes without error."""
time.sleep(2)
try:
print(f"\n✅ reviews_print_fields output:")
self.scraper.reviews_print_fields(self.app_id, ["userName", "score"], count=self.count, sort=self.sort, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_reviews_print_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"reviews_print_fields raised unexpected {e}")
def test_reviews_print_all(self):
"""Test reviews_print_all executes without error."""
time.sleep(2)
try:
print(f"\n✅ reviews_print_all output:")
self.scraper.reviews_print_all(self.app_id, count=self.count, sort=self.sort, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_reviews_print_all: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"reviews_print_all raised unexpected {e}")
if __name__ == '__main__':
unittest.main()
================================================
FILE: tests/test_search_methods.py
================================================
"""
Unit tests for Search Methods
"""
import unittest
import time
import warnings
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import GPlayScraperError, NetworkError, RateLimitError
class TestSearchMethods(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.scraper = GPlayScraper() # Initialize scraper
cls.query = "social media" # Search query
cls.count = 10 # Number of items to fetch
cls.lang = "en" # Language
cls.country = "us" # Country
def test_search_analyze(self):
"""Test search_analyze returns list of results"""
time.sleep(2)
try:
result = self.scraper.search_analyze(self.query, count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
self.assertGreater(len(result), 0)
self.assertIn('title', result[0])
print(f"\n✅ Search results for '{self.query}' ({len(result)} apps):")
for i, app in enumerate(result[:3]): # Show first 3 results
print(f" {i+1}. {app.get('title', 'N/A')} - {app.get('developer', 'N/A')}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_search_analyze: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_search_get_field(self):
"""Test search_get_field returns list of field values"""
time.sleep(2)
try:
result = self.scraper.search_get_field(self.query, "title", count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
self.assertGreater(len(result), 0)
print(f"\n✅ App titles from search ({len(result)} results):")
for i, title in enumerate(result[:3]):
print(f" {i+1}. {title}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_search_get_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_search_get_fields(self):
"""Test search_get_fields returns list of dictionaries"""
time.sleep(2)
fields = ["title", "score"]
try:
result = self.scraper.search_get_fields(self.query, fields, count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
self.assertGreater(len(result), 0)
for field in fields:
self.assertIn(field, result[0])
print(f"\n✅ Multiple fields from search ({len(result)} results):")
for i, app in enumerate(result[:3]):
print(f" {i+1}. {app.get('title', 'N/A')} - Score: {app.get('score', 'N/A')}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_search_get_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_search_print_field(self):
"""Test search_print_field executes without error"""
time.sleep(2)
try:
print(f"\n✅ search_print_field output for '{self.query}':")
self.scraper.search_print_field(self.query, "title", count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_search_print_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"search_print_field raised unexpected {type(e).__name__}: {e}")
def test_search_print_fields(self):
"""Test search_print_fields executes without error"""
time.sleep(2)
try:
print(f"\n✅ search_print_fields output for '{self.query}':")
self.scraper.search_print_fields(self.query, ["title", "score"], count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_search_print_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"search_print_fields raised unexpected {type(e).__name__}: {e}")
def test_search_print_all(self):
"""Test search_print_all executes without error"""
time.sleep(2)
try:
print(f"\n✅ search_print_all output for '{self.query}':")
self.scraper.search_print_all(self.query, count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_search_print_all: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"search_print_all raised unexpected {type(e).__name__}: {e}")
if __name__ == '__main__':
unittest.main()
================================================
FILE: tests/test_similar_methods.py
================================================
"""
Unit tests for Similar Methods
"""
import unittest
import time
import warnings
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import GPlayScraperError, NetworkError, RateLimitError
class TestSimilarMethods(unittest.TestCase):
"""Test suite for similar methods (find related apps)."""
def setUp(self):
"""Set up test fixtures before each test method."""
self.scraper = GPlayScraper() # Initialize scraper
self.app_id = "com.whatsapp" # WhatsApp app ID for testing
self.count = 10 # Number of items to fetch
self.lang = "en" # Language
self.country = "us" # Country
def test_similar_analyze(self):
"""Test similar_analyze returns list of similar apps."""
time.sleep(2)
try:
result = self.scraper.similar_analyze(self.app_id, count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
self.assertGreater(len(result), 0)
print(f"\n✅ Similar apps to {self.app_id} ({len(result)} apps):")
for i, app in enumerate(result[:3]): # Show first 3 apps
print(f" {i+1}. {app.get('title', 'N/A')} - {app.get('developer', 'N/A')}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_similar_analyze: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_similar_get_field(self):
"""Test similar_get_field returns list of field values."""
time.sleep(2)
try:
result = self.scraper.similar_get_field(self.app_id, "title", count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
print(f"\n✅ Similar app titles ({len(result)} apps):")
for i, title in enumerate(result[:3]):
print(f" {i+1}. {title}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_similar_get_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_similar_get_fields(self):
"""Test similar_get_fields returns list of dictionaries."""
time.sleep(2)
try:
result = self.scraper.similar_get_fields(self.app_id, ["title", "score"], count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
print(f"\n✅ Similar app fields ({len(result)} apps):")
for i, app in enumerate(result[:3]):
print(f" {i+1}. {app.get('title', 'N/A')} - {app.get('score', 'N/A')} stars")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_similar_get_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_similar_print_field(self):
"""Test similar_print_field executes without error."""
time.sleep(2)
try:
print(f"\n✅ similar_print_field output:")
self.scraper.similar_print_field(self.app_id, "title", count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_similar_print_field: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"similar_print_field raised unexpected {e}")
def test_similar_print_fields(self):
"""Test similar_print_fields executes without error."""
time.sleep(2)
try:
print(f"\n✅ similar_print_fields output:")
self.scraper.similar_print_fields(self.app_id, ["title", "score"], count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_similar_print_fields: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"similar_print_fields raised unexpected {e}")
def test_similar_print_all(self):
"""Test similar_print_all executes without error."""
time.sleep(2)
try:
print(f"\n✅ similar_print_all output:")
self.scraper.similar_print_all(self.app_id, count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_similar_print_all: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"similar_print_all raised unexpected {e}")
if __name__ == '__main__':
unittest.main()
================================================
FILE: tests/test_suggest_methods.py
================================================
"""
Unit tests for Suggest Methods
"""
import unittest
import time
import warnings
from gplay_scraper import GPlayScraper
from gplay_scraper.exceptions import GPlayScraperError, NetworkError, RateLimitError
class TestSuggestMethods(unittest.TestCase):
"""Test suite for suggest methods (search suggestions)."""
def setUp(self):
"""Set up test fixtures before each test method."""
self.scraper = GPlayScraper() # Initialize scraper
self.term = "fitness" # Search term for testing
self.count = 10 # Number of items to fetch
self.lang = "en" # Language
self.country = "us" # Country
def test_suggest_analyze(self):
"""Test suggest_analyze returns list of suggestions."""
time.sleep(2)
try:
result = self.scraper.suggest_analyze(self.term, count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, list)
if result:
self.assertGreater(len(result), 0)
print(f"\n✅ Search suggestions for '{self.term}' ({len(result)} suggestions):")
for i, suggestion in enumerate(result):
print(f" {i+1}. {suggestion}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_suggest_analyze: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_suggest_nested(self):
"""Test suggest_nested returns nested suggestions."""
time.sleep(2)
try:
result = self.scraper.suggest_nested(self.term, count=self.count, lang=self.lang, country=self.country)
self.assertIsInstance(result, dict)
if result:
print(f"\n✅ Nested suggestions for '{self.term}':")
for key, suggestions in list(result.items())[:2]: # Show first 2 nested
print(f" '{key}' -> {suggestions[:3]}")
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_suggest_nested: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
def test_suggest_print_nested(self):
"""Test suggest_print_nested executes without error."""
time.sleep(2)
try:
print(f"\n✅ suggest_print_nested output:")
self.scraper.suggest_print_nested(self.term, count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_suggest_print_nested: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"suggest_print_nested raised unexpected {e}")
def test_suggest_print_all(self):
"""Test suggest_print_all executes without error."""
time.sleep(2)
try:
print(f"\n✅ suggest_print_all output:")
self.scraper.suggest_print_all(self.term, count=self.count, lang=self.lang, country=self.country)
except (NetworkError, RateLimitError, GPlayScraperError) as e:
warnings.warn(f"Network/Rate limit error in test_suggest_print_all: {e}")
self.skipTest(f"Skipping due to network/rate limit: {e}")
except Exception as e:
self.fail(f"suggest_print_all raised unexpected {e}")
if __name__ == '__main__':
unittest.main()
