Repository: nashsu/FreeAskInternet
Branch: main
Commit: 3fa58d17e908
Files: 10
Total size: 95.7 KB
Directory structure:
gitextract_mcwgdhv6/
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── docker-compose.yaml
├── free_ask_internet.py
├── requirements.txt
├── searxng/
│ ├── settings.yml
│ └── uwsgi.ini
└── server.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
__pycache__/
*.py[cod]
*$py.class
================================================
FILE: Dockerfile
================================================
FROM python:3.9.15
WORKDIR /app
COPY requirements.txt /app
RUN pip3 install -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r requirements.txt --no-cache-dir
COPY . /app
EXPOSE 8000
ENTRYPOINT ["python3"]
CMD ["server.py"]
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# FreeAskInternet
## 🎉🎉🎉 Yeah we have a logo now! 🎉🎉🎉
> Running www.perplexity.ai like app complete FREE, LOCAL, PRIVATE and NO GPU NEED on any computer
> [!IMPORTANT]
> **If you are unable to use this project normally, it is most likely due to issues with your internet connection or your IP, you need free internet connection to use this project normally. 如果您无法正常使用此项目，很可能是由于您的 IP 存在问题，或者你不能自由访问互联网。**
## What is FreeAskInternet
FreeAskInternet is a completely free, private and locally running search aggregator & answer generate using LLM, Without GPU needed. The user can ask a question and the system will use searxng to make a multi engine search and combine the search result to the ChatGPT3.5 LLM and generate the answer based on search results. All process running locally and No GPU or OpenAI or Google API keys are needed.
## Features
- 🈚️ Completely FREE (no need for any API keys)
- 💻 Completely LOCAL (no GPU need, any computer can run )
- 🔐 Completely PRIVATE (all thing running locally, using custom llm)
- 👻 Runs WITHOUT LLM Hardware (NO GPU NEED!)
- 🤩 Using Free ChatGPT3.5 / Qwen / Kimi / ZhipuAI(GLM) API (NO API keys need! Thx OpenAI)
- 🐵 Custom LLM(ollama,llama.cpp) support, Yes we love ollama!
- 🚀 Fast and easy to deploy with Docker Compose
- 🌐 Web and Mobile friendly interface, designed for Web Search enhanced AI Chat, allowing for easy access from any device.
## Screenshots
1. index:
2. Search based AI Chat:
3. Multi LLM models and custom LLM like ollama support:
## How It Works?
1. System get user input question in FreeAskInternet UI interface( running locally), and call searxng (running locally) to make search on multi search engine.
2. crawl search result links content and pass to ChatGPT3.5 / Kimi / Qwen / ZhipuAI / ollama (by using custom llm), ask LLM to answer user question based on this contents as references.
3. Stream the answer to Chat UI.
4. We support custom LLM setting, so theoretically infinite llm support.
## Status
This project is still in its very early days. Expect some bugs.
### Run the latest release
```bash
git clone https://github.com/nashsu/FreeAskInternet.git
cd ./FreeAskInternet
docker-compose up -d
```
🎉 You should now be able to open the web interface on http://localhost:3000. Nothing else is exposed by default.( For old web interface, accessing http://localhost:3030)
## How to get and set Kimi / Qwen / ZhipuAI Token?
How to get Token?
We are using [https://github.com/LLM-Red-Team](https://github.com/LLM-Red-Team) projects to provide those service, you can reference to their readme.
Reference : [https://github.com/LLM-Red-Team/kimi-free-api](https://github.com/LLM-Red-Team/kimi-free-api)
## How to using custom LLM like ollama? (Yes we love ollama)
1. start ollama serve
```bash
export OLLAMA_HOST=0.0.0.0
ollama serve
```
2. set ollama url in setting:
You MUST using your computer's ip address, not localhost/127.0.0.1, because in docker you can't access this address.
The model name is the model you want to serve by ollama.
ollama model Reference : [https://ollama.com/library](https://ollama.com/library)
### How to update to latest
```bash
cd ./FreeAskInternet
git pull
docker compose down
docker compose rm backend
docker compose rm free_ask_internet_ui
docker image rm nashsu/free_ask_internet
docker image rm nashsu/free_ask_internet_ui
docker-compose up -d
```
## Credits
- ChatGPT-Next-Web : [https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web)
- FreeGPT35: [https://github.com/missuo/FreeGPT35](https://github.com/missuo/FreeGPT35)
- Kimi\Qwen\ZhipuAI [https://github.com/LLM-Red-Team](https://github.com/LLM-Red-Team)
- searxng: [https://github.com/searxng/searxng](https://github.com/searxng/searxng)
## Special thanks to our logo designer
[AdlerMurcus](https://github.com/AdlerMurcus)
<a href="https://github.com/AdlerMurcus">
<img src="https://avatars.githubusercontent.com/u/40649955?v=4" width="100" height="100" class="avatar avatar-user width-full border color-bg-default"/>
</a>
## License
Apache-2.0 license
## Star History
[](https://star-history.com/#nashsu/FreeAskInternet&Date)
================================================
FILE: docker-compose.yaml
================================================
services:
backend:
image: docker.io/nashsu/free_ask_internet:latest
depends_on:
- llm-freegpt35
restart: on-failure
freeaskinternet-ui:
image: docker.io/nashsu/free_ask_internet_ui:latest
ports:
- "3000:80"
environment:
BACKEND_HOST: "backend:8000"
depends_on:
- backend
restart: always
chatgpt-next-web:
image: yidadaa/chatgpt-next-web
ports:
- "3030:3000"
environment:
OPENAI_API_KEY: "FreeAskInternet"
# CODE: "FreeAskInternet" # 如果你想要设置页面的访问密码，请修改这里
BASE_URL: "http://backend:8000"
CUSTOM_MODELS: "-all,+gpt-3.5-turbo"
depends_on:
- llm-freegpt35
restart: always
llm-freegpt35:
image: missuo/freegpt35:latest
restart: always
llm-kimi:
image: vinlic/kimi-free-api:latest
restart: always
environment:
- TZ=Asia/Shanghai
llm-glm4:
image: vinlic/glm-free-api:latest
restart: always
environment:
- TZ=Asia/Shanghai
llm-qwen:
image: vinlic/qwen-free-api:latest
restart: always
environment:
- TZ=Asia/Shanghai
searxng:
image: docker.io/searxng/searxng:latest
volumes:
- ./searxng:/etc/searxng:rw
environment:
- SEARXNG_BASE_URL=https://${SEARXNG_HOSTNAME:-localhost}/
cap_drop:
- ALL
cap_add:
- CHOWN
- SETGID
- SETUID
logging:
driver: 'json-file'
options:
max-size: '1m'
max-file: '1'
restart: always
================================================
FILE: free_ask_internet.py
================================================
# -*- coding: utf-8 -*-
import json
import os
from pprint import pprint
import requests
import trafilatura
from trafilatura import bare_extraction
from concurrent.futures import ThreadPoolExecutor
import concurrent
import requests
import openai
import time
from datetime import datetime
from urllib.parse import urlparse
import tldextract
import platform
import urllib.parse
def extract_url_content(url):
downloaded = trafilatura.fetch_url(url)
content = trafilatura.extract(downloaded)
return {"url":url, "content":content}
def search_web_ref(query:str, debug=False):
content_list = []
try:
safe_string = urllib.parse.quote_plus(":all !general " + query)
response = requests.get('http://searxng:8080?q=' + safe_string + '&format=json')
response.raise_for_status()
search_results = response.json()
if debug:
print("JSON Response:")
pprint(search_results)
pedding_urls = []
conv_links = []
if search_results.get('results'):
for item in search_results.get('results')[0:9]:
name = item.get('title')
snippet = item.get('content')
url = item.get('url')
pedding_urls.append(url)
if url:
url_parsed = urlparse(url)
domain = url_parsed.netloc
icon_url = url_parsed.scheme + '://' + url_parsed.netloc + '/favicon.ico'
site_name = tldextract.extract(url).domain
conv_links.append({
'site_name':site_name,
'icon_url':icon_url,
'title':name,
'url':url,
'snippet':snippet
})
results = []
futures = []
executor = ThreadPoolExecutor(max_workers=10)
for url in pedding_urls:
futures.append(executor.submit(extract_url_content,url))
try:
for future in futures:
res = future.result(timeout=5)
results.append(res)
except concurrent.futures.TimeoutError:
print("任务执行超时")
executor.shutdown(wait=False,cancel_futures=True)
for content in results:
if content and content.get('content'):
item_dict = {
"url":content.get('url'),
"content": content.get('content'),
"length":len(content.get('content'))
}
content_list.append(item_dict)
if debug:
print("URL: {}".format(url))
print("=================")
return conv_links,content_list
except Exception as ex:
raise ex
def gen_prompt(question,content_list, lang="zh-CN", context_length_limit=11000,debug=False):
limit_len = (context_length_limit - 2000)
if len(question) > limit_len:
question = question[0:limit_len]
ref_content = [ item.get("content") for item in content_list]
answer_language = ' Simplified Chinese '
if lang == "zh-CN":
answer_language = ' Simplified Chinese '
if lang == "zh-TW":
answer_language = ' Traditional Chinese '
if lang == "en-US":
answer_language = ' English '
if len(ref_content) > 0:
if False:
prompts = '''
您是一位由 nash_su 开发的大型语言人工智能助手。您将被提供一个用户问题，并需要撰写一个清晰、简洁且准确的答案。提供了一组与问题相关的上下文，每个都以[[citation:x]]这样的编号开头，x代表一个数字。请在适当的情况下在句子末尾引用上下文。答案必须正确、精确，并以专家的中立和职业语气撰写。请将答案限制在2000个标记内。不要提供与问题无关的信息，也不要重复。如果给出的上下文信息不足，请在相关主题后写上“信息缺失：”。请按照引用编号[citation:x]的格式在答案中对应部分引用上下文。如果一句话源自多个上下文，请列出所有相关的引用编号，例如[citation:3][citation:5]，不要将引用集中在最后返回，而是在答案对应部分列出。除非是代码、特定的名称或引用编号，答案的语言应与问题相同。以下是上下文的内容集：
''' + "\n\n" + "```"
ref_index = 1
for ref_text in ref_content:
prompts = prompts + "\n\n" + " [citation:{}] ".format(str(ref_index)) + ref_text
ref_index += 1
if len(prompts) >= limit_len:
prompts = prompts[0:limit_len]
prompts = prompts + '''
```
记住，不要一字不差的重复上下文内容. 回答必须使用简体中文，如果回答很长，请尽量结构化、分段落总结。请按照引用编号[citation:x]的格式在答案中对应部分引用上下文。如果一句话源自多个上下文，请列出所有相关的引用编号，例如[citation:3][citation:5]，不要将引用集中在最后返回，而是在答案对应部分列出。下面是用户问题：
''' + question
else:
prompts = '''
You are a large language AI assistant develop by nash_su. You are given a user question, and please write clean, concise and accurate answer to the question. You will be given a set of related contexts to the question, each starting with a reference number like [[citation:x]], where x is a number. Please use the context and cite the context at the end of each sentence if applicable.
Your answer must be correct, accurate and written by an expert using an unbiased and professional tone. Please limit to 1024 tokens. Do not give any information that is not related to the question, and do not repeat. Say "information is missing on" followed by the related topic, if the given context do not provide sufficient information.
Please cite the contexts with the reference numbers, in the format [citation:x]. If a sentence comes from multiple contexts, please list all applicable citations, like [citation:3][citation:5]. Other than code and specific names and citations, your answer must be written in the same language as the question.
Here are the set of contexts:
''' + "\n\n" + "```"
ref_index = 1
for ref_text in ref_content:
prompts = prompts + "\n\n" + " [citation:{}] ".format(str(ref_index)) + ref_text
ref_index += 1
if len(prompts) >= limit_len:
prompts = prompts[0:limit_len]
prompts = prompts + '''
```
Above is the reference contexts. Remember, don't repeat the context word for word. Answer in ''' + answer_language + '''. If the response is lengthy, structure it in paragraphs and summarize where possible. Cite the context using the format [citation:x] where x is the reference number. If a sentence originates from multiple contexts, list all relevant citation numbers, like [citation:3][citation:5]. Don't cluster the citations at the end but include them in the answer where they correspond.
Remember, don't blindly repeat the contexts verbatim. And here is the user question:
''' + question
else:
prompts = question
if debug:
print(prompts)
print("总长度："+ str(len(prompts)))
return prompts
def chat(prompt, model:str,llm_auth_token:str,llm_base_url:str,using_custom_llm=False,stream=True, debug=False):
openai.base_url = "http://127.0.0.1:3040/v1/"
if model == "gpt3.5":
openai.base_url = "http://llm-freegpt35:3040/v1/"
if model == "kimi":
openai.base_url = "http://llm-kimi:8000/v1/"
if model == "glm4":
openai.base_url = "http://llm-glm4:8000/v1/"
if model == "qwen":
openai.base_url = "http://llm-qwen:8000/v1/"
if llm_auth_token == '':
llm_auth_token = "CUSTOM"
openai.api_key = llm_auth_token
if using_custom_llm:
openai.base_url = llm_base_url
openai.api_key = llm_auth_token
total_content = ""
for chunk in openai.chat.completions.create(
model=model,
messages=[{
"role": "user",
"content": prompt
}],
stream=True,
max_tokens=1024,temperature=0.2
):
stream_resp = chunk.dict()
token = stream_resp["choices"][0]["delta"].get("content", "")
if token:
total_content += token
yield token
if debug:
print(total_content)
def ask_internet(query:str, debug=False):
content_list = search_web_ref(query,debug=debug)
if debug:
print(content_list)
prompt = gen_prompt(query,content_list,context_length_limit=6000,debug=debug)
total_token = ""
for token in chat(prompt=prompt):
# for token in daxianggpt.chat(prompt=prompt):
if token:
total_token += token
yield token
yield "\n\n"
# 是否返回参考资料
if True:
yield "---"
yield "\n"
yield "参考资料:\n"
count = 1
for url_content in content_list:
url = url_content.get('url')
yield "*[{}. {}]({})*".format(str(count),url,url )
yield "\n"
count += 1
================================================
FILE: requirements.txt
================================================
annotated-types==0.6.0
anyio==4.3.0
certifi==2024.2.2
charset-normalizer==3.3.2
click==8.1.7
courlan==1.0.0
dateparser==1.2.0
distro==1.9.0
exceptiongroup==1.2.0
fastapi==0.110.1
filelock==3.13.3
h11==0.14.0
htmldate==1.8.0
httpcore==1.0.5
httpx==0.27.0
idna==3.6
jusText==3.0.0
langcodes==3.3.0
lxml==5.1.1
openai==1.16.2
pydantic==2.6.4
pydantic_core==2.16.3
python-dateutil==2.9.0.post0
pytz==2024.1
regex==2023.12.25
requests==2.31.0
requests-file==2.0.0
six==1.16.0
sniffio==1.3.1
sse-starlette==2.0.0
starlette==0.37.2
tld==0.13
tldextract==5.1.2
tqdm==4.66.2
trafilatura==1.8.1
typing_extensions==4.10.0
tzlocal==5.2
urllib3==2.2.1
uvicorn==0.29.0
================================================
FILE: searxng/settings.yml
================================================
general:
# Debug mode, only for development. Is overwritten by ${SEARXNG_DEBUG}
debug: false
# displayed name
instance_name: "free_ask_internet"
# For example: https://example.com/privacy
privacypolicy_url: false
# use true to use your own donation page written in searx/info/en/donate.md
# use false to disable the donation link
donation_url: false
# mailto:contact@example.com
contact_url: false
# record stats
enable_metrics: true
brand:
new_issue_url: https://github.com/searxng/searxng/issues/new
docs_url: https://docs.searxng.org/
public_instances: https://searx.space
wiki_url: https://github.com/searxng/searxng/wiki
issue_url: https://github.com/searxng/searxng/issues
# custom:
# maintainer: "Jon Doe"
# # Custom entries in the footer: [title]: [link]
# links:
# Uptime: https://uptime.searxng.org/history/darmarit-org
# About: "https://searxng.org"
search:
# Filter results. 0: None, 1: Moderate, 2: Strict
safe_search: 0
# Existing autocomplete backends: "dbpedia", "duckduckgo", "google", "yandex", "mwmbl",
# "seznam", "startpage", "stract", "swisscows", "qwant", "wikipedia" - leave blank to turn it off
# by default.
autocomplete: ""
# minimun characters to type before autocompleter starts
# Default search language - leave blank to detect from browser information or
# use codes from 'languages.py'
default_lang: "auto"
ban_time_on_fail: 1
max_ban_time_on_fail: 120
suspended_times:
SearxEngineAccessDenied: 86400
SearxEngineCaptcha: 86400
SearxEngineTooManyRequests: 3600
cf_SearxEngineCaptcha: 1296000
cf_SearxEngineAccessDenied: 86400
recaptcha_SearxEngineCaptcha: 604800
formats:
- html
- json
server:
# Is overwritten by ${SEARXNG_PORT} and ${SEARXNG_BIND_ADDRESS}
port: 8080
bind_address: "0.0.0.0"
# public URL of the instance, to ensure correct inbound links. Is overwritten
# by ${SEARXNG_URL}.
base_url: http://localhost:/ # "http://example.com/location"
limiter: false # rate limit the number of request on the instance, block some bots
public_instance: false # enable features designed only for public instances
# If your instance owns a /etc/searxng/settings.yml file, then set the following
# values there.
secret_key: "f884dadd9e6b23ea6c073a031b18daa0288ff1d82a1c5514089f0ecb32ac7a54" # Is overwritten by ${SEARXNG_SECRET}
# Proxying image results through searx
image_proxy: false
# 1.0 and 1.1 are supported
http_protocol_version: "1.0"
# POST queries are more secure as they don't show up in history but may cause
# problems when using Firefox containers
method: "POST"
default_http_headers:
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Robots-Tag: noindex, nofollow
Referrer-Policy: no-referrer
redis:
# URL to connect redis database. Is overwritten by ${SEARXNG_REDIS_URL}.
# https://docs.searxng.org/admin/settings/settings_redis.html#settings-redis
url: false
ui:
# Custom static path - leave it blank if you didn't change
static_path: ""
static_use_hash: false
# Custom templates path - leave it blank if you didn't change
templates_path: ""
# query_in_title: When true, the result page's titles contains the query
# it decreases the privacy, since the browser can records the page titles.
query_in_title: false
# infinite_scroll: When true, automatically loads the next page when scrolling to bottom of the current page.
infinite_scroll: false
# ui theme
default_theme: simple
# center the results ?
center_alignment: false
# URL prefix of the internet archive, don't forget trailing slash (if needed).
# cache_url: "https://webcache.googleusercontent.com/search?q=cache:"
# Default interface locale - leave blank to detect from browser information or
# use codes from the 'locales' config section
default_locale: ""
# Open result links in a new tab by default
# results_on_new_tab: false
theme_args:
# style of simple theme: auto, light, dark
simple_style: auto
# Perform search immediately if a category selected.
# Disable to select multiple categories at once and start the search manually.
search_on_category_select: true
# Hotkeys: default or vim
hotkeys: default
# Lock arbitrary settings on the preferences page. To find the ID of the user
# setting you want to lock, check the ID of the form on the page "preferences".
#
# preferences:
# lock:
# - language
# - autocomplete
# - method
# - query_in_title
# searx supports result proxification using an external service:
# https://github.com/asciimoo/morty uncomment below section if you have running
# morty proxy the key is base64 encoded (keep the !!binary notation)
# Note: since commit af77ec3, morty accepts a base64 encoded key.
#
# result_proxy:
# url: http://127.0.0.1:3000/
# # the key is a base64 encoded string, the YAML !!binary prefix is optional
# key: !!binary "your_morty_proxy_key"
# # [true|false] enable the "proxy" button next to each result
# proxify_results: true
# communication with search engines
#
outgoing:
# default timeout in seconds, can be override by engine
request_timeout: 3.0
# the maximum timeout in seconds
# max_request_timeout: 10.0
# suffix of searx_useragent, could contain information like an email address
# to the administrator
useragent_suffix: ""
# The maximum number of concurrent connections that may be established.
pool_connections: 100
# Allow the connection pool to maintain keep-alive connections below this
# point.
pool_maxsize: 20
# See https://www.python-httpx.org/http2/
enable_http2: true
# uncomment below section if you want to use a custom server certificate
# see https://www.python-httpx.org/advanced/#changing-the-verification-defaults
# and https://www.python-httpx.org/compatibility/#ssl-configuration
# verify: ~/.mitmproxy/mitmproxy-ca-cert.cer
#
# uncomment below section if you want to use a proxyq see: SOCKS proxies
# https://2.python-requests.org/en/latest/user/advanced/#proxies
# are also supported: see
# https://2.python-requests.org/en/latest/user/advanced/#socks
#
# proxies:
# all://:
# - http://proxy1:8080
# - http://proxy2:8080
#
# using_tor_proxy: true
#
# Extra seconds to add in order to account for the time taken by the proxy
#
# extra_proxy_timeout: 10
#
# uncomment below section only if you have more than one network interface
# which can be the source of outgoing search requests
#
# source_ips:
# - 1.1.1.1
# - 1.1.1.2
# - fe80::/126
# External plugin configuration, for more details see
# https://docs.searxng.org/dev/plugins.html
#
# plugins:
# - plugin1
# - plugin2
# - ...
# Comment or un-comment plugin to activate / deactivate by default.
#
# enabled_plugins:
# # these plugins are enabled if nothing is configured ..
# - 'Hash plugin'
# - 'Self Information'
# - 'Tracker URL remover'
# - 'Ahmia blacklist' # activation depends on outgoing.using_tor_proxy
# # these plugins are disabled if nothing is configured ..
# - 'Hostname replace' # see hostname_replace configuration below
# - 'Open Access DOI rewrite'
# - 'Tor check plugin'
# # Read the docs before activate: auto-detection of the language could be
# # detrimental to users expectations / users can activate the plugin in the
# # preferences if they want.
# - 'Autodetect search language'
# Configuration of the "Hostname replace" plugin:
#
# hostname_replace:
# '(.*\.)?youtube\.com$': 'invidious.example.com'
# '(.*\.)?youtu\.be$': 'invidious.example.com'
# '(.*\.)?youtube-noocookie\.com$': 'yotter.example.com'
# '(.*\.)?reddit\.com$': 'teddit.example.com'
# '(.*\.)?redd\.it$': 'teddit.example.com'
# '(www\.)?twitter\.com$': 'nitter.example.com'
# # to remove matching host names from result list, set value to false
# 'spam\.example\.com': false
checker:
# disable checker when in debug mode
off_when_debug: true
# use "scheduling: false" to disable scheduling
# scheduling: interval or int
# to activate the scheduler:
# * uncomment "scheduling" section
# * add "cache2 = name=searxngcache,items=2000,blocks=2000,blocksize=4096,bitmap=1"
# to your uwsgi.ini
# scheduling:
# start_after: [300, 1800] # delay to start the first run of the checker
# every: [86400, 90000] # how often the checker runs
# additional tests: only for the YAML anchors (see the engines section)
#
additional_tests:
rosebud: &test_rosebud
matrix:
query: rosebud
lang: en
result_container:
- not_empty
- ['one_title_contains', 'citizen kane']
test:
- unique_results
android: &test_android
matrix:
query: ['android']
lang: ['en', 'de', 'fr', 'zh-CN']
result_container:
- not_empty
- ['one_title_contains', 'google']
test:
- unique_results
# tests: only for the YAML anchors (see the engines section)
tests:
infobox: &tests_infobox
infobox:
matrix:
query: ["linux", "new york", "bbc"]
result_container:
- has_infobox
categories_as_tabs:
general:
images:
videos:
news:
map:
music:
it:
science:
files:
social media:
engines:
- name: 9gag
engine: 9gag
shortcut: 9g
disabled: true
- name: annas archive
engine: annas_archive
disabled: true
shortcut: aa
# - name: annas articles
# engine: annas_archive
# shortcut: aaa
# # https://docs.searxng.org/dev/engines/online/annas_archive.html
# aa_content: 'magazine' # book_fiction, book_unknown, book_nonfiction, book_comic
# aa_ext: 'pdf' # pdf, epub, ..
# aa_sort: oldest' # newest, oldest, largest, smallest
- name: apk mirror
engine: apkmirror
timeout: 4.0
shortcut: apkm
disabled: true
- name: apple app store
engine: apple_app_store
shortcut: aps
disabled: true
# Requires Tor
- name: ahmia
engine: ahmia
categories: onions
enable_http: true
shortcut: ah
- name: anaconda
engine: xpath
paging: true
first_page_num: 0
search_url: https://anaconda.org/search?q={query}&page={pageno}
results_xpath: //tbody/tr
url_xpath: ./td/h5/a[last()]/@href
title_xpath: ./td/h5
content_xpath: ./td[h5]/text()
categories: it
timeout: 6.0
shortcut: conda
disabled: true
- name: arch linux wiki
engine: archlinux
shortcut: al
- name: artic
engine: artic
shortcut: arc
timeout: 4.0
- name: arxiv
engine: arxiv
shortcut: arx
timeout: 4.0
- name: ask
engine: ask
shortcut: ask
disabled: true
# tmp suspended: dh key too small
# - name: base
# engine: base
# shortcut: bs
- name: bandcamp
engine: bandcamp
shortcut: bc
categories: music
- name: wikipedia
engine: wikipedia
shortcut: wp
# add "list" to the array to get results in the results list
display_type: ["infobox"]
base_url: 'https://{language}.wikipedia.org/'
categories: [general]
- name: bilibili
engine: bilibili
shortcut: bil
disabled: true
- name: bing
engine: bing
shortcut: bi
disabled: true
- name: bing images
engine: bing_images
shortcut: bii
- name: bing news
engine: bing_news
shortcut: bin
- name: bing videos
engine: bing_videos
shortcut: biv
- name: bitbucket
engine: xpath
paging: true
search_url: https://bitbucket.org/repo/all/{pageno}?name={query}
url_xpath: //article[@class="repo-summary"]//a[@class="repo-link"]/@href
title_xpath: //article[@class="repo-summary"]//a[@class="repo-link"]
content_xpath: //article[@class="repo-summary"]/p
categories: [it, repos]
timeout: 4.0
disabled: true
shortcut: bb
about:
website: https://bitbucket.org/
wikidata_id: Q2493781
official_api_documentation: https://developer.atlassian.com/bitbucket
use_official_api: false
require_api_key: false
results: HTML
- name: bpb
engine: bpb
shortcut: bpb
disabled: true
- name: btdigg
engine: btdigg
shortcut: bt
disabled: true
- name: ccc-tv
engine: xpath
paging: false
search_url: https://media.ccc.de/search/?q={query}
url_xpath: //div[@class="caption"]/h3/a/@href
title_xpath: //div[@class="caption"]/h3/a/text()
content_xpath: //div[@class="caption"]/h4/@title
categories: videos
disabled: true
shortcut: c3tv
about:
website: https://media.ccc.de/
wikidata_id: Q80729951
official_api_documentation: https://github.com/voc/voctoweb
use_official_api: false
require_api_key: false
results: HTML
# We don't set language: de here because media.ccc.de is not just
# for a German audience. It contains many English videos and many
# German videos have English subtitles.
- name: openverse
engine: openverse
categories: images
shortcut: opv
- name: chefkoch
engine: chefkoch
shortcut: chef
# to show premium or plus results too:
# skip_premium: false
# - name: core.ac.uk
# engine: core
# categories: science
# shortcut: cor
# # get your API key from: https://core.ac.uk/api-keys/register/
# api_key: 'unset'
- name: cppreference
engine: cppreference
shortcut: cpp
paging: false
disabled: true
- name: crossref
engine: crossref
shortcut: cr
timeout: 30
disabled: true
- name: crowdview
engine: json_engine
shortcut: cv
categories: general
paging: false
search_url: https://crowdview-next-js.onrender.com/api/search-v3?query={query}
results_query: results
url_query: link
title_query: title
content_query: snippet
disabled: true
about:
website: https://crowdview.ai/
- name: yep
engine: yep
shortcut: yep
categories: general
search_type: web
disabled: true
- name: yep images
engine: yep
shortcut: yepi
categories: images
search_type: images
disabled: true
- name: yep news
engine: yep
shortcut: yepn
categories: news
search_type: news
disabled: true
- name: curlie
engine: xpath
shortcut: cl
categories: general
disabled: true
paging: true
lang_all: ''
search_url: https://curlie.org/search?q={query}&lang={lang}&start={pageno}&stime=92452189
page_size: 20
results_xpath: //div[@id="site-list-content"]/div[@class="site-item"]
url_xpath: ./div[@class="title-and-desc"]/a/@href
title_xpath: ./div[@class="title-and-desc"]/a/div
content_xpath: ./div[@class="title-and-desc"]/div[@class="site-descr"]
about:
website: https://curlie.org/
wikidata_id: Q60715723
use_official_api: false
require_api_key: false
results: HTML
- name: currency
engine: currency_convert
categories: general
shortcut: cc
- name: bahnhof
engine: json_engine
search_url: https://www.bahnhof.de/api/stations/search/{query}
url_prefix: https://www.bahnhof.de/
url_query: slug
title_query: name
content_query: state
shortcut: bf
disabled: true
about:
website: https://www.bahn.de
wikidata_id: Q22811603
use_official_api: false
require_api_key: false
results: JSON
language: de
tests:
bahnhof:
matrix:
query: berlin
lang: en
result_container:
- not_empty
- ['one_title_contains', 'Berlin Hauptbahnhof']
test:
- unique_results
- name: deezer
engine: deezer
shortcut: dz
disabled: true
- name: destatis
engine: destatis
shortcut: destat
disabled: true
- name: deviantart
engine: deviantart
shortcut: da
timeout: 3.0
- name: ddg definitions
engine: duckduckgo_definitions
shortcut: ddd
weight: 2
disabled: true
tests: *tests_infobox
# cloudflare protected
# - name: digbt
# engine: digbt
# shortcut: dbt
# timeout: 6.0
# disabled: true
- name: docker hub
engine: docker_hub
shortcut: dh
categories: [it, packages]
- name: erowid
engine: xpath
paging: true
first_page_num: 0
page_size: 30
search_url: https://www.erowid.org/search.php?q={query}&s={pageno}
url_xpath: //dl[@class="results-list"]/dt[@class="result-title"]/a/@href
title_xpath: //dl[@class="results-list"]/dt[@class="result-title"]/a/text()
content_xpath: //dl[@class="results-list"]/dd[@class="result-details"]
categories: []
shortcut: ew
disabled: true
about:
website: https://www.erowid.org/
wikidata_id: Q1430691
official_api_documentation:
use_official_api: false
require_api_key: false
results: HTML
# - name: elasticsearch
# shortcut: es
# engine: elasticsearch
# base_url: http://localhost:9200
# username: elastic
# password: changeme
# index: my-index
# # available options: match, simple_query_string, term, terms, custom
# query_type: match
# # if query_type is set to custom, provide your query here
# #custom_query_json: {"query":{"match_all": {}}}
# #show_metadata: false
# disabled: true
- name: wikidata
engine: wikidata
shortcut: wd
timeout: 3.0
weight: 2
# add "list" to the array to get results in the results list
display_type: ["infobox"]
tests: *tests_infobox
categories: [general]
- name: duckduckgo
engine: duckduckgo
shortcut: ddg
- name: duckduckgo images
engine: duckduckgo_extra
categories: [images, web]
ddg_category: images
shortcut: ddi
disabled: true
- name: duckduckgo videos
engine: duckduckgo_extra
categories: [videos, web]
ddg_category: videos
shortcut: ddv
disabled: true
- name: duckduckgo news
engine: duckduckgo_extra
categories: [news, web]
ddg_category: news
shortcut: ddn
disabled: true
- name: duckduckgo weather
engine: duckduckgo_weather
shortcut: ddw
disabled: true
- name: apple maps
engine: apple_maps
shortcut: apm
disabled: true
timeout: 5.0
- name: emojipedia
engine: emojipedia
timeout: 4.0
shortcut: em
disabled: true
- name: tineye
engine: tineye
shortcut: tin
timeout: 9.0
disabled: true
- name: etymonline
engine: xpath
paging: true
search_url: https://etymonline.com/search?page={pageno}&q={query}
url_xpath: //a[contains(@class, "word__name--")]/@href
title_xpath: //a[contains(@class, "word__name--")]
content_xpath: //section[contains(@class, "word__defination")]
first_page_num: 1
shortcut: et
categories: [dictionaries]
about:
website: https://www.etymonline.com/
wikidata_id: Q1188617
official_api_documentation:
use_official_api: false
require_api_key: false
results: HTML
# - name: ebay
# engine: ebay
# shortcut: eb
# base_url: 'https://www.ebay.com'
# disabled: true
# timeout: 5
- name: 1x
engine: www1x
shortcut: 1x
timeout: 3.0
disabled: true
- name: fdroid
engine: fdroid
shortcut: fd
disabled: true
- name: flickr
categories: images
shortcut: fl
# You can use the engine using the official stable API, but you need an API
# key, see: https://www.flickr.com/services/apps/create/
# engine: flickr
# api_key: 'apikey' # required!
# Or you can use the html non-stable engine, activated by default
engine: flickr_noapi
- name: free software directory
engine: mediawiki
shortcut: fsd
categories: [it, software wikis]
base_url: https://directory.fsf.org/
search_type: title
timeout: 5.0
disabled: true
about:
website: https://directory.fsf.org/
wikidata_id: Q2470288
# - name: freesound
# engine: freesound
# shortcut: fnd
# disabled: true
# timeout: 15.0
# API key required, see: https://freesound.org/docs/api/overview.html
# api_key: MyAPIkey
- name: frinkiac
engine: frinkiac
shortcut: frk
disabled: true
- name: fyyd
engine: fyyd
shortcut: fy
timeout: 8.0
disabled: true
- name: genius
engine: genius
shortcut: gen
- name: gentoo
engine: gentoo
shortcut: ge
timeout: 10.0
- name: gitlab
engine: json_engine
paging: true
search_url: https://gitlab.com/api/v4/projects?search={query}&page={pageno}
url_query: web_url
title_query: name_with_namespace
content_query: description
page_size: 20
categories: [it, repos]
shortcut: gl
timeout: 10.0
disabled: true
about:
website: https://about.gitlab.com/
wikidata_id: Q16639197
official_api_documentation: https://docs.gitlab.com/ee/api/
use_official_api: false
require_api_key: false
results: JSON
- name: github
engine: github
shortcut: gh
# This a Gitea service. If you would like to use a different instance,
# change codeberg.org to URL of the desired Gitea host. Or you can create a
# new engine by copying this and changing the name, shortcut and search_url.
- name: codeberg
engine: json_engine
search_url: https://codeberg.org/api/v1/repos/search?q={query}&limit=10
url_query: html_url
title_query: name
content_query: description
categories: [it, repos]
shortcut: cb
disabled: true
about:
website: https://codeberg.org/
wikidata_id:
official_api_documentation: https://try.gitea.io/api/swagger
use_official_api: false
require_api_key: false
results: JSON
- name: goodreads
engine: goodreads
shortcut: good
timeout: 4.0
disabled: true
- name: google
engine: google
shortcut: go
# additional_tests:
# android: *test_android
- name: google images
engine: google_images
shortcut: goi
# additional_tests:
# android: *test_android
# dali:
# matrix:
# query: ['Dali Christ']
# lang: ['en', 'de', 'fr', 'zh-CN']
# result_container:
# - ['one_title_contains', 'Salvador']
- name: google news
engine: google_news
shortcut: gon
# additional_tests:
# android: *test_android
- name: google videos
engine: google_videos
shortcut: gov
# additional_tests:
# android: *test_android
- name: google scholar
engine: google_scholar
shortcut: gos
- name: google play apps
engine: google_play
categories: [files, apps]
shortcut: gpa
play_categ: apps
disabled: true
- name: google play movies
engine: google_play
categories: videos
shortcut: gpm
play_categ: movies
disabled: true
- name: material icons
engine: material_icons
categories: images
shortcut: mi
disabled: true
- name: gpodder
engine: json_engine
shortcut: gpod
timeout: 4.0
paging: false
search_url: https://gpodder.net/search.json?q={query}
url_query: url
title_query: title
content_query: description
page_size: 19
categories: music
disabled: true
about:
website: https://gpodder.net
wikidata_id: Q3093354
official_api_documentation: https://gpoddernet.readthedocs.io/en/latest/api/
use_official_api: false
requires_api_key: false
results: JSON
- name: habrahabr
engine: xpath
paging: true
search_url: https://habr.com/en/search/page{pageno}/?q={query}
results_xpath: //article[contains(@class, "tm-articles-list__item")]
url_xpath: .//a[@class="tm-title__link"]/@href
title_xpath: .//a[@class="tm-title__link"]
content_xpath: .//div[contains(@class, "article-formatted-body")]
categories: it
timeout: 4.0
disabled: true
shortcut: habr
about:
website: https://habr.com/
wikidata_id: Q4494434
official_api_documentation: https://habr.com/en/docs/help/api/
use_official_api: false
require_api_key: false
results: HTML
- name: hackernews
engine: hackernews
shortcut: hn
disabled: true
- name: hoogle
engine: xpath
search_url: https://hoogle.haskell.org/?hoogle={query}
results_xpath: '//div[@class="result"]'
title_xpath: './/div[@class="ans"]//a'
url_xpath: './/div[@class="ans"]//a/@href'
content_xpath: './/div[@class="from"]'
page_size: 20
categories: [it, packages]
shortcut: ho
about:
website: https://hoogle.haskell.org/
wikidata_id: Q34010
official_api_documentation: https://hackage.haskell.org/api
use_official_api: false
require_api_key: false
results: JSON
- name: imdb
engine: imdb
shortcut: imdb
timeout: 6.0
disabled: true
- name: imgur
engine: imgur
shortcut: img
disabled: true
- name: ina
engine: ina
shortcut: in
timeout: 6.0
disabled: true
- name: invidious
engine: invidious
# Instanes will be selected randomly, see https://api.invidious.io/ for
# instances that are stable (good uptime) and close to you.
base_url:
- https://invidious.io.lol
- https://invidious.fdn.fr
- https://yt.artemislena.eu
- https://invidious.tiekoetter.com
- https://invidious.flokinet.to
- https://vid.puffyan.us
- https://invidious.privacydev.net
- https://inv.tux.pizza
shortcut: iv
timeout: 3.0
disabled: true
- name: jisho
engine: jisho
shortcut: js
timeout: 3.0
disabled: true
- name: kickass
engine: kickass
base_url:
- https://kickasstorrents.to
- https://kickasstorrents.cr
- https://kickasstorrent.cr
- https://kickass.sx
- https://kat.am
shortcut: kc
timeout: 4.0
- name: lemmy communities
engine: lemmy
lemmy_type: Communities
shortcut: leco
- name: lemmy users
engine: lemmy
network: lemmy communities
lemmy_type: Users
shortcut: leus
- name: lemmy posts
engine: lemmy
network: lemmy communities
lemmy_type: Posts
shortcut: lepo
- name: lemmy comments
engine: lemmy
network: lemmy communities
lemmy_type: Comments
shortcut: lecom
- name: library genesis
engine: xpath
# search_url: https://libgen.is/search.php?req={query}
search_url: https://libgen.rs/search.php?req={query}
url_xpath: //a[contains(@href,"book/index.php?md5")]/@href
title_xpath: //a[contains(@href,"book/")]/text()[1]
content_xpath: //td/a[1][contains(@href,"=author")]/text()
categories: files
timeout: 7.0
disabled: true
shortcut: lg
about:
website: https://libgen.fun/
wikidata_id: Q22017206
official_api_documentation:
use_official_api: false
require_api_key: false
results: HTML
- name: z-library
engine: zlibrary
shortcut: zlib
categories: files
timeout: 7.0
- name: library of congress
engine: loc
shortcut: loc
categories: images
- name: lingva
engine: lingva
shortcut: lv
# set lingva instance in url, by default it will use the official instance
# url: https://lingva.thedaviddelta.com
- name: lobste.rs
engine: xpath
search_url: https://lobste.rs/search?q={query}&what=stories&order=relevance
results_xpath: //li[contains(@class, "story")]
url_xpath: .//a[@class="u-url"]/@href
title_xpath: .//a[@class="u-url"]
content_xpath: .//a[@class="domain"]
categories: it
shortcut: lo
timeout: 5.0
disabled: true
about:
website: https://lobste.rs/
wikidata_id: Q60762874
official_api_documentation:
use_official_api: false
require_api_key: false
results: HTML
- name: azlyrics
shortcut: lyrics
engine: xpath
timeout: 4.0
disabled: true
categories: [music, lyrics]
paging: true
search_url: https://search.azlyrics.com/search.php?q={query}&w=lyrics&p={pageno}
url_xpath: //td[@class="text-left visitedlyr"]/a/@href
title_xpath: //span/b/text()
content_xpath: //td[@class="text-left visitedlyr"]/a/small
about:
website: https://azlyrics.com
wikidata_id: Q66372542
official_api_documentation:
use_official_api: false
require_api_key: false
results: HTML
- name: mastodon users
engine: mastodon
mastodon_type: accounts
base_url: https://mastodon.social
shortcut: mau
- name: mastodon hashtags
engine: mastodon
mastodon_type: hashtags
base_url: https://mastodon.social
shortcut: mah
# - name: matrixrooms
# engine: mrs
# # https://docs.searxng.org/dev/engines/online/mrs.html
# # base_url: https://mrs-api-host
# shortcut: mtrx
# disabled: true
- name: mdn
shortcut: mdn
engine: json_engine
categories: [it]
paging: true
search_url: https://developer.mozilla.org/api/v1/search?q={query}&page={pageno}
results_query: documents
url_query: mdn_url
url_prefix: https://developer.mozilla.org
title_query: title
content_query: summary
about:
website: https://developer.mozilla.org
wikidata_id: Q3273508
official_api_documentation: null
use_official_api: false
require_api_key: false
results: JSON
- name: metacpan
engine: metacpan
shortcut: cpan
disabled: true
number_of_results: 20
# - name: meilisearch
# engine: meilisearch
# shortcut: mes
# enable_http: true
# base_url: http://localhost:7700
# index: my-index
- name: mixcloud
engine: mixcloud
shortcut: mc
# MongoDB engine
# Required dependency: pymongo
# - name: mymongo
# engine: mongodb
# shortcut: md
# exact_match_only: false
# host: '127.0.0.1'
# port: 27017
# enable_http: true
# results_per_page: 20
# database: 'business'
# collection: 'reviews' # name of the db collection
# key: 'name' # key in the collection to search for
- name: mozhi
engine: mozhi
base_url:
- https://mozhi.aryak.me
- https://translate.bus-hit.me
- https://nyc1.mz.ggtyler.dev
# mozhi_engine: google - see https://mozhi.aryak.me for supported engines
timeout: 4.0
shortcut: mz
disabled: true
- name: mwmbl
engine: mwmbl
# api_url: https://api.mwmbl.org
shortcut: mwm
disabled: true
- name: npm
engine: npm
shortcut: npm
timeout: 5.0
disabled: true
- name: nyaa
engine: nyaa
shortcut: nt
disabled: true
- name: mankier
engine: json_engine
search_url: https://www.mankier.com/api/v2/mans/?q={query}
results_query: results
url_query: url
title_query: name
content_query: description
categories: it
shortcut: man
about:
website: https://www.mankier.com/
official_api_documentation: https://www.mankier.com/api
use_official_api: true
require_api_key: false
results: JSON
# read https://docs.searxng.org/dev/engines/online/mullvad_leta.html
# - name: mullvadleta
# engine: mullvad_leta
# use_cache: true # Only 100 non-cache searches per day, suggested only for private instances
# search_url: https://leta.mullvad.net
# categories: [general, web]
# shortcut: ml
- name: odysee
engine: odysee
shortcut: od
disabled: true
- name: openairedatasets
engine: json_engine
paging: true
search_url: https://api.openaire.eu/search/datasets?format=json&page={pageno}&size=10&title={query}
results_query: response/results/result
url_query: metadata/oaf:entity/oaf:result/children/instance/webresource/url/$
title_query: metadata/oaf:entity/oaf:result/title/$
content_query: metadata/oaf:entity/oaf:result/description/$
content_html_to_text: true
categories: "science"
shortcut: oad
timeout: 5.0
about:
website: https://www.openaire.eu/
wikidata_id: Q25106053
official_api_documentation: https://api.openaire.eu/
use_official_api: false
require_api_key: false
results: JSON
- name: openairepublications
engine: json_engine
paging: true
search_url: https://api.openaire.eu/search/publications?format=json&page={pageno}&size=10&title={query}
results_query: response/results/result
url_query: metadata/oaf:entity/oaf:result/children/instance/webresource/url/$
title_query: metadata/oaf:entity/oaf:result/title/$
content_query: metadata/oaf:entity/oaf:result/description/$
content_html_to_text: true
categories: science
shortcut: oap
timeout: 5.0
about:
website: https://www.openaire.eu/
wikidata_id: Q25106053
official_api_documentation: https://api.openaire.eu/
use_official_api: false
require_api_key: false
results: JSON
# - name: opensemanticsearch
# engine: opensemantic
# shortcut: oss
# base_url: 'http://localhost:8983/solr/opensemanticsearch/'
- name: openstreetmap
engine: openstreetmap
shortcut: osm
- name: openrepos
engine: xpath
paging: true
search_url: https://openrepos.net/search/node/{query}?page={pageno}
url_xpath: //li[@class="search-result"]//h3[@class="title"]/a/@href
title_xpath: //li[@class="search-result"]//h3[@class="title"]/a
content_xpath: //li[@class="search-result"]//div[@class="search-snippet-info"]//p[@class="search-snippet"]
categories: files
timeout: 4.0
disabled: true
shortcut: or
about:
website: https://openrepos.net/
wikidata_id:
official_api_documentation:
use_official_api: false
require_api_key: false
results: HTML
- name: packagist
engine: json_engine
paging: true
search_url: https://packagist.org/search.json?q={query}&page={pageno}
results_query: results
url_query: url
title_query: name
content_query: description
categories: [it, packages]
disabled: true
timeout: 5.0
shortcut: pack
about:
website: https://packagist.org
wikidata_id: Q108311377
official_api_documentation: https://packagist.org/apidoc
use_official_api: true
require_api_key: false
results: JSON
- name: pdbe
engine: pdbe
shortcut: pdb
# Hide obsolete PDB entries. Default is not to hide obsolete structures
# hide_obsolete: false
- name: photon
engine: photon
shortcut: ph
- name: pinterest
engine: pinterest
shortcut: pin
- name: piped
engine: piped
shortcut: ppd
categories: videos
piped_filter: videos
timeout: 3.0
# URL to use as link and for embeds
frontend_url: https://srv.piped.video
# Instance will be selected randomly, for more see https://piped-instances.kavin.rocks/
backend_url:
- https://pipedapi.kavin.rocks
- https://pipedapi-libre.kavin.rocks
- https://pipedapi.adminforge.de
- name: piped.music
engine: piped
network: piped
shortcut: ppdm
categories: music
piped_filter: music_songs
timeout: 3.0
- name: piratebay
engine: piratebay
shortcut: tpb
# You may need to change this URL to a proxy if piratebay is blocked in your
# country
url: https://thepiratebay.org/
timeout: 3.0
- name: pixiv
shortcut: pv
engine: pixiv
disabled: true
inactive: true
pixiv_image_proxies:
- pximg.example.org
# A proxy is required to load the images. Hosting an image proxy server
# for Pixiv:
# --> https://codeberg.org/VnPower/PixivFE/wiki/Hosting-an-image-proxy-server-for-Pixiv
# Proxies from public instances. Ask the public instances owners if they
# agree to receive traffic from SearXNG!
# --> https://codeberg.org/VnPower/PixivFE#instances
# --> https://github.com/searxng/searxng/pull/3192#issuecomment-1941095047
# image proxy of https://pixiv.cat
# - https://i.pixiv.cat
# image proxy of https://www.pixiv.pics
# - https://pximg.cocomi.eu.org
# image proxy of https://pixivfe.exozy.me
# - https://pximg.exozy.me
# image proxy of https://pixivfe.ducks.party
# - https://pixiv.ducks.party
# image proxy of https://pixiv.perennialte.ch
# - https://pximg.perennialte.ch
- name: podcastindex
engine: podcastindex
shortcut: podcast
# Required dependency: psychopg2
# - name: postgresql
# engine: postgresql
# database: postgres
# username: postgres
# password: postgres
# limit: 10
# query_str: 'SELECT * from my_table WHERE my_column = %(query)s'
# shortcut : psql
- name: presearch
engine: presearch
search_type: search
categories: [general, web]
shortcut: ps
timeout: 4.0
disabled: true
- name: presearch images
engine: presearch
network: presearch
search_type: images
categories: [images, web]
timeout: 4.0
shortcut: psimg
disabled: true
- name: presearch videos
engine: presearch
network: presearch
search_type: videos
categories: [general, web]
timeout: 4.0
shortcut: psvid
disabled: true
- name: presearch news
engine: presearch
network: presearch
search_type: news
categories: [news, web]
timeout: 4.0
shortcut: psnews
disabled: true
- name: pub.dev
engine: xpath
shortcut: pd
search_url: https://pub.dev/packages?q={query}&page={pageno}
paging: true
results_xpath: //div[contains(@class,"packages-item")]
url_xpath: ./div/h3/a/@href
title_xpath: ./div/h3/a
content_xpath: ./div/div/div[contains(@class,"packages-description")]/span
categories: [packages, it]
timeout: 3.0
disabled: true
first_page_num: 1
about:
website: https://pub.dev/
official_api_documentation: https://pub.dev/help/api
use_official_api: false
require_api_key: false
results: HTML
- name: pubmed
engine: pubmed
shortcut: pub
timeout: 3.0
- name: pypi
shortcut: pypi
engine: pypi
- name: qwant
qwant_categ: web
engine: qwant
shortcut: qw
categories: [general, web]
additional_tests:
rosebud: *test_rosebud
- name: qwant news
qwant_categ: news
engine: qwant
shortcut: qwn
categories: news
network: qwant
- name: qwant images
qwant_categ: images
engine: qwant
shortcut: qwi
categories: [images, web]
network: qwant
- name: qwant videos
qwant_categ: videos
engine: qwant
shortcut: qwv
categories: [videos, web]
network: qwant
# - name: library
# engine: recoll
# shortcut: lib
# base_url: 'https://recoll.example.org/'
# search_dir: ''
# mount_prefix: /export
# dl_prefix: 'https://download.example.org'
# timeout: 30.0
# categories: files
# disabled: true
# - name: recoll library reference
# engine: recoll
# base_url: 'https://recoll.example.org/'
# search_dir: reference
# mount_prefix: /export
# dl_prefix: 'https://download.example.org'
# shortcut: libr
# timeout: 30.0
# categories: files
# disabled: true
- name: radio browser
engine: radio_browser
shortcut: rb
- name: reddit
engine: reddit
shortcut: re
page_size: 25
- name: rottentomatoes
engine: rottentomatoes
shortcut: rt
disabled: true
# Required dependency: redis
# - name: myredis
# shortcut : rds
# engine: redis_server
# exact_match_only: false
# host: '127.0.0.1'
# port: 6379
# enable_http: true
# password: ''
# db: 0
# tmp suspended: bad certificate
# - name: scanr structures
# shortcut: scs
# engine: scanr_structures
# disabled: true
- name: sepiasearch
engine: sepiasearch
shortcut: sep
- name: soundcloud
engine: soundcloud
shortcut: sc
- name: stackoverflow
engine: stackexchange
shortcut: st
api_site: 'stackoverflow'
categories: [it, q&a]
- name: askubuntu
engine: stackexchange
shortcut: ubuntu
api_site: 'askubuntu'
categories: [it, q&a]
- name: internetarchivescholar
engine: internet_archive_scholar
shortcut: ias
timeout: 15.0
- name: superuser
engine: stackexchange
shortcut: su
api_site: 'superuser'
categories: [it, q&a]
- name: searchcode code
engine: searchcode_code
shortcut: scc
disabled: true
# - name: searx
# engine: searx_engine
# shortcut: se
# instance_urls :
# - http://127.0.0.1:8888/
# - ...
# disabled: true
- name: semantic scholar
engine: semantic_scholar
disabled: true
shortcut: se
# Spotify needs API credentials
# - name: spotify
# engine: spotify
# shortcut: stf
# api_client_id: *******
# api_client_secret: *******
# - name: solr
# engine: solr
# shortcut: slr
# base_url: http://localhost:8983
# collection: collection_name
# sort: '' # sorting: asc or desc
# field_list: '' # comma separated list of field names to display on the UI
# default_fields: '' # default field to query
# query_fields: '' # query fields
# enable_http: true
# - name: springer nature
# engine: springer
# # get your API key from: https://dev.springernature.com/signup
# # working API key, for test & debug: "a69685087d07eca9f13db62f65b8f601"
# api_key: 'unset'
# shortcut: springer
# timeout: 15.0
- name: startpage
engine: startpage
shortcut: sp
timeout: 6.0
disabled: true
additional_tests:
rosebud: *test_rosebud
- name: tokyotoshokan
engine: tokyotoshokan
shortcut: tt
timeout: 6.0
disabled: true
- name: solidtorrents
engine: solidtorrents
shortcut: solid
timeout: 4.0
base_url:
- https://solidtorrents.to
- https://bitsearch.to
# For this demo of the sqlite engine download:
# https://liste.mediathekview.de/filmliste-v2.db.bz2
# and unpack into searx/data/filmliste-v2.db
# Query to test: "!demo concert"
#
# - name: demo
# engine: sqlite
# shortcut: demo
# categories: general
# result_template: default.html
# database: searx/data/filmliste-v2.db
# query_str: >-
# SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title,
# COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url,
# description AS content
# FROM film
# WHERE title LIKE :wildcard OR description LIKE :wildcard
# ORDER BY duration DESC
- name: tagesschau
engine: tagesschau
# when set to false, display URLs from Tagesschau, and not the actual source
# (e.g. NDR, WDR, SWR, HR, ...)
use_source_url: true
shortcut: ts
disabled: true
- name: tmdb
engine: xpath
paging: true
categories: movies
search_url: https://www.themoviedb.org/search?page={pageno}&query={query}
results_xpath: //div[contains(@class,"movie") or contains(@class,"tv")]//div[contains(@class,"card")]
url_xpath: .//div[contains(@class,"poster")]/a/@href
thumbnail_xpath: .//img/@src
title_xpath: .//div[contains(@class,"title")]//h2
content_xpath: .//div[contains(@class,"overview")]
shortcut: tm
disabled: true
# Requires Tor
- name: torch
engine: xpath
paging: true
search_url:
http://xmh57jrknzkhv6y3ls3ubitzfqnkrwxhopf5aygthi7d6rplyvk3noyd.onion/cgi-bin/omega/omega?P={query}&DEFAULTOP=and
results_xpath: //table//tr
url_xpath: ./td[2]/a
title_xpath: ./td[2]/b
content_xpath: ./td[2]/small
categories: onions
enable_http: true
shortcut: tch
# torznab engine lets you query any torznab compatible indexer. Using this
# engine in combination with Jackett opens the possibility to query a lot of
# public and private indexers directly from SearXNG. More details at:
# https://docs.searxng.org/dev/engines/online/torznab.html
#
# - name: Torznab EZTV
# engine: torznab
# shortcut: eztv
# base_url: http://localhost:9117/api/v2.0/indexers/eztv/results/torznab
# enable_http: true # if using localhost
# api_key: xxxxxxxxxxxxxxx
# show_magnet_links: true
# show_torrent_files: false
# # https://github.com/Jackett/Jackett/wiki/Jackett-Categories
# torznab_categories: # optional
# - 2000
# - 5000
# tmp suspended - too slow, too many errors
# - name: urbandictionary
# engine : xpath
# search_url : https://www.urbandictionary.com/define.php?term={query}
# url_xpath : //*[@class="word"]/@href
# title_xpath : //*[@class="def-header"]
# content_xpath: //*[@class="meaning"]
# shortcut: ud
- name: unsplash
engine: unsplash
shortcut: us
- name: yandex music
engine: yandex_music
shortcut: ydm
disabled: true
# https://yandex.com/support/music/access.html
inactive: true
- name: yahoo
engine: yahoo
shortcut: yh
disabled: true
- name: yahoo news
engine: yahoo_news
shortcut: yhn
- name: youtube
shortcut: yt
# You can use the engine using the official stable API, but you need an API
# key See: https://console.developers.google.com/project
#
# engine: youtube_api
# api_key: 'apikey' # required!
#
# Or you can use the html non-stable engine, activated by default
engine: youtube_noapi
- name: dailymotion
engine: dailymotion
shortcut: dm
- name: vimeo
engine: vimeo
shortcut: vm
- name: wiby
engine: json_engine
paging: true
search_url: https://wiby.me/json/?q={query}&p={pageno}
url_query: URL
title_query: Title
content_query: Snippet
categories: [general, web]
shortcut: wib
disabled: true
about:
website: https://wiby.me/
- name: alexandria
engine: json_engine
shortcut: alx
categories: general
paging: true
search_url: https://api.alexandria.org/?a=1&q={query}&p={pageno}
results_query: results
title_query: title
url_query: url
content_query: snippet
timeout: 1.5
disabled: true
about:
website: https://alexandria.org/
official_api_documentation: https://github.com/alexandria-org/alexandria-api/raw/master/README.md
use_official_api: true
require_api_key: false
results: JSON
- name: wikibooks
engine: mediawiki
weight: 0.5
shortcut: wb
categories: [general, wikimedia]
base_url: "https://{language}.wikibooks.org/"
search_type: text
disabled: true
about:
website: https://www.wikibooks.org/
wikidata_id: Q367
- name: wikinews
engine: mediawiki
shortcut: wn
categories: [news, wikimedia]
base_url: "https://{language}.wikinews.org/"
search_type: text
srsort: create_timestamp_desc
about:
website: https://www.wikinews.org/
wikidata_id: Q964
- name: wikiquote
engine: mediawiki
weight: 0.5
shortcut: wq
categories: [general, wikimedia]
base_url: "https://{language}.wikiquote.org/"
search_type: text
disabled: true
additional_tests:
rosebud: *test_rosebud
about:
website: https://www.wikiquote.org/
wikidata_id: Q369
- name: wikisource
engine: mediawiki
weight: 0.5
shortcut: ws
categories: [general, wikimedia]
base_url: "https://{language}.wikisource.org/"
search_type: text
disabled: true
about:
website: https://www.wikisource.org/
wikidata_id: Q263
- name: wikispecies
engine: mediawiki
shortcut: wsp
categories: [general, science, wikimedia]
base_url: "https://species.wikimedia.org/"
search_type: text
disabled: true
about:
website: https://species.wikimedia.org/
wikidata_id: Q13679
tests:
wikispecies:
matrix:
query: "Campbell, L.I. et al. 2011: MicroRNAs"
lang: en
result_container:
- not_empty
- ['one_title_contains', 'Tardigrada']
test:
- unique_results
- name: wiktionary
engine: mediawiki
shortcut: wt
categories: [dictionaries, wikimedia]
base_url: "https://{language}.wiktionary.org/"
search_type: text
about:
website: https://www.wiktionary.org/
wikidata_id: Q151
- name: wikiversity
engine: mediawiki
weight: 0.5
shortcut: wv
categories: [general, wikimedia]
base_url: "https://{language}.wikiversity.org/"
search_type: text
disabled: true
about:
website: https://www.wikiversity.org/
wikidata_id: Q370
- name: wikivoyage
engine: mediawiki
weight: 0.5
shortcut: wy
categories: [general, wikimedia]
base_url: "https://{language}.wikivoyage.org/"
search_type: text
disabled: true
about:
website: https://www.wikivoyage.org/
wikidata_id: Q373
- name: wikicommons.images
engine: wikicommons
shortcut: wc
categories: images
number_of_results: 10
- name: wolframalpha
shortcut: wa
# You can use the engine using the official stable API, but you need an API
# key. See: https://products.wolframalpha.com/api/
#
# engine: wolframalpha_api
# api_key: ''
#
# Or you can use the html non-stable engine, activated by default
engine: wolframalpha_noapi
timeout: 6.0
categories: general
disabled: true
- name: dictzone
engine: dictzone
shortcut: dc
- name: mymemory translated
engine: translated
shortcut: tl
timeout: 5.0
# You can use without an API key, but you are limited to 1000 words/day
# See: https://mymemory.translated.net/doc/usagelimits.php
# api_key: ''
# Required dependency: mysql-connector-python
# - name: mysql
# engine: mysql_server
# database: mydatabase
# username: user
# password: pass
# limit: 10
# query_str: 'SELECT * from mytable WHERE fieldname=%(query)s'
# shortcut: mysql
- name: 1337x
engine: 1337x
shortcut: 1337x
disabled: true
- name: duden
engine: duden
shortcut: du
disabled: true
- name: seznam
shortcut: szn
engine: seznam
disabled: true
# - name: deepl
# engine: deepl
# shortcut: dpl
# # You can use the engine using the official stable API, but you need an API key
# # See: https://www.deepl.com/pro-api?cta=header-pro-api
# api_key: '' # required!
# timeout: 5.0
# disabled: true
- name: mojeek
shortcut: mjk
engine: xpath
paging: true
categories: [general, web]
search_url: https://www.mojeek.com/search?q={query}&s={pageno}&lang={lang}&lb={lang}
results_xpath: //ul[@class="results-standard"]/li/a[@class="ob"]
url_xpath: ./@href
title_xpath: ../h2/a
content_xpath: ..//p[@class="s"]
suggestion_xpath: //div[@class="top-info"]/p[@class="top-info spell"]/em/a
first_page_num: 0
page_size: 10
max_page: 100
disabled: true
about:
website: https://www.mojeek.com/
wikidata_id: Q60747299
official_api_documentation: https://www.mojeek.com/services/api.html/
use_official_api: false
require_api_key: false
results: HTML
- name: moviepilot
engine: moviepilot
shortcut: mp
disabled: true
- name: naver
shortcut: nvr
categories: [general, web]
engine: xpath
paging: true
search_url: https://search.naver.com/search.naver?where=webkr&sm=osp_hty&ie=UTF-8&query={query}&start={pageno}
url_xpath: //a[@class="link_tit"]/@href
title_xpath: //a[@class="link_tit"]
content_xpath: //div[@class="total_dsc_wrap"]/a
first_page_num: 1
page_size: 10
disabled: true
about:
website: https://www.naver.com/
wikidata_id: Q485639
official_api_documentation: https://developers.naver.com/docs/nmt/examples/
use_official_api: false
require_api_key: false
results: HTML
language: ko
- name: rubygems
shortcut: rbg
engine: xpath
paging: true
search_url: https://rubygems.org/search?page={pageno}&query={query}
results_xpath: /html/body/main/div/a[@class="gems__gem"]
url_xpath: ./@href
title_xpath: ./span/h2
content_xpath: ./span/p
suggestion_xpath: /html/body/main/div/div[@class="search__suggestions"]/p/a
first_page_num: 1
categories: [it, packages]
disabled: true
about:
website: https://rubygems.org/
wikidata_id: Q1853420
official_api_documentation: https://guides.rubygems.org/rubygems-org-api/
use_official_api: false
require_api_key: false
results: HTML
- name: peertube
engine: peertube
shortcut: ptb
paging: true
# alternatives see: https://instances.joinpeertube.org/instances
# base_url: https://tube.4aem.com
categories: videos
disabled: true
timeout: 6.0
- name: mediathekviewweb
engine: mediathekviewweb
shortcut: mvw
disabled: true
- name: yacy
engine: yacy
categories: general
search_type: text
base_url: https://yacy.searchlab.eu
shortcut: ya
disabled: true
# required if you aren't using HTTPS for your local yacy instance
# https://docs.searxng.org/dev/engines/online/yacy.html
# enable_http: true
# timeout: 3.0
# search_mode: 'global'
- name: yacy images
engine: yacy
categories: images
search_type: image
base_url: https://yacy.searchlab.eu
shortcut: yai
disabled: true
- name: rumble
engine: rumble
shortcut: ru
base_url: https://rumble.com/
paging: true
categories: videos
disabled: true
- name: livespace
engine: livespace
shortcut: ls
categories: videos
disabled: true
timeout: 5.0
- name: wordnik
engine: wordnik
shortcut: def
base_url: https://www.wordnik.com/
categories: [dictionaries]
timeout: 5.0
- name: woxikon.de synonyme
engine: xpath
shortcut: woxi
categories: [dictionaries]
timeout: 5.0
disabled: true
search_url: https://synonyme.woxikon.de/synonyme/{query}.php
url_xpath: //div[@class="upper-synonyms"]/a/@href
content_xpath: //div[@class="synonyms-list-group"]
title_xpath: //div[@class="upper-synonyms"]/a
no_result_for_http_status: [404]
about:
website: https://www.woxikon.de/
wikidata_id: # No Wikidata ID
use_official_api: false
require_api_key: false
results: HTML
language: de
- name: seekr news
engine: seekr
shortcut: senews
categories: news
seekr_category: news
disabled: true
- name: seekr images
engine: seekr
network: seekr news
shortcut: seimg
categories: images
seekr_category: images
disabled: true
- name: seekr videos
engine: seekr
network: seekr news
shortcut: sevid
categories: videos
seekr_category: videos
disabled: true
- name: sjp.pwn
engine: sjp
shortcut: sjp
base_url: https://sjp.pwn.pl/
timeout: 5.0
disabled: true
- name: stract
engine: stract
shortcut: str
disabled: true
- name: svgrepo
engine: svgrepo
shortcut: svg
timeout: 10.0
disabled: true
- name: tootfinder
engine: tootfinder
shortcut: toot
- name: voidlinux
engine: voidlinux
shortcut: void
disabled: true
- name: wallhaven
engine: wallhaven
# api_key: abcdefghijklmnopqrstuvwxyz
shortcut: wh
# wikimini: online encyclopedia for children
# The fulltext and title parameter is necessary for Wikimini because
# sometimes it will not show the results and redirect instead
- name: wikimini
engine: xpath
shortcut: wkmn
search_url: https://fr.wikimini.org/w/index.php?search={query}&title=Sp%C3%A9cial%3ASearch&fulltext=Search
url_xpath: //li/div[@class="mw-search-result-heading"]/a/@href
title_xpath: //li//div[@class="mw-search-result-heading"]/a
content_xpath: //li/div[@class="searchresult"]
categories: general
disabled: true
about:
website: https://wikimini.org/
wikidata_id: Q3568032
use_official_api: false
require_api_key: false
results: HTML
language: fr
- name: wttr.in
engine: wttr
shortcut: wttr
timeout: 9.0
- name: yummly
engine: yummly
shortcut: yum
disabled: true
- name: brave
engine: brave
shortcut: br
time_range_support: true
paging: true
categories: [general, web]
brave_category: search
# brave_spellcheck: true
- name: brave.images
engine: brave
network: brave
shortcut: brimg
categories: [images, web]
brave_category: images
- name: brave.videos
engine: brave
network: brave
shortcut: brvid
categories: [videos, web]
brave_category: videos
- name: brave.news
engine: brave
network: brave
shortcut: brnews
categories: news
brave_category: news
# - name: brave.goggles
# engine: brave
# network: brave
# shortcut: brgog
# time_range_support: true
# paging: true
# categories: [general, web]
# brave_category: goggles
# Goggles: # required! This should be a URL ending in .goggle
- name: lib.rs
shortcut: lrs
engine: lib_rs
disabled: true
- name: sourcehut
shortcut: srht
engine: xpath
paging: true
search_url: https://sr.ht/projects?page={pageno}&search={query}
results_xpath: (//div[@class="event-list"])[1]/div[@class="event"]
url_xpath: ./h4/a[2]/@href
title_xpath: ./h4/a[2]
content_xpath: ./p
first_page_num: 1
categories: [it, repos]
disabled: true
about:
website: https://sr.ht
wikidata_id: Q78514485
official_api_documentation: https://man.sr.ht/
use_official_api: false
require_api_key: false
results: HTML
- name: goo
shortcut: goo
engine: xpath
paging: true
search_url: https://search.goo.ne.jp/web.jsp?MT={query}&FR={pageno}0
url_xpath: //div[@class="result"]/p[@class='title fsL1']/a/@href
title_xpath: //div[@class="result"]/p[@class='title fsL1']/a
content_xpath: //p[contains(@class,'url fsM')]/following-sibling::p
first_page_num: 0
categories: [general, web]
disabled: true
timeout: 4.0
about:
website: https://search.goo.ne.jp
wikidata_id: Q249044
use_official_api: false
require_api_key: false
results: HTML
language: ja
- name: bt4g
engine: bt4g
shortcut: bt4g
- name: pkg.go.dev
engine: pkg_go_dev
shortcut: pgo
disabled: true
# Doku engine lets you access to any Doku wiki instance:
# A public one or a privete/corporate one.
# - name: ubuntuwiki
# engine: doku
# shortcut: uw
# base_url: 'https://doc.ubuntu-fr.org'
# Be careful when enabling this engine if you are
# running a public instance. Do not expose any sensitive
# information. You can restrict access by configuring a list
# of access tokens under tokens.
# - name: git grep
# engine: command
# command: ['git', 'grep', '{{QUERY}}']
# shortcut: gg
# tokens: []
# disabled: true
# delimiter:
# chars: ':'
# keys: ['filepath', 'code']
# Be careful when enabling this engine if you are
# running a public instance. Do not expose any sensitive
# information. You can restrict access by configuring a list
# of access tokens under tokens.
# - name: locate
# engine: command
# command: ['locate', '{{QUERY}}']
# shortcut: loc
# tokens: []
# disabled: true
# delimiter:
# chars: ' '
# keys: ['line']
# Be careful when enabling this engine if you are
# running a public instance. Do not expose any sensitive
# information. You can restrict access by configuring a list
# of access tokens under tokens.
# - name: find
# engine: command
# command: ['find', '.', '-name', '{{QUERY}}']
# query_type: path
# shortcut: fnd
# tokens: []
# disabled: true
# delimiter:
# chars: ' '
# keys: ['line']
# Be careful when enabling this engine if you are
# running a public instance. Do not expose any sensitive
# information. You can restrict access by configuring a list
# of access tokens under tokens.
# - name: pattern search in files
# engine: command
# command: ['fgrep', '{{QUERY}}']
# shortcut: fgr
# tokens: []
# disabled: true
# delimiter:
# chars: ' '
# keys: ['line']
# Be careful when enabling this engine if you are
# running a public instance. Do not expose any sensitive
# information. You can restrict access by configuring a list
# of access tokens under tokens.
# - name: regex search in files
# engine: command
# command: ['grep', '{{QUERY}}']
# shortcut: gr
# tokens: []
# disabled: true
# delimiter:
# chars: ' '
# keys: ['line']
doi_resolvers:
oadoi.org: 'https://oadoi.org/'
doi.org: 'https://doi.org/'
doai.io: 'https://dissem.in/'
sci-hub.se: 'https://sci-hub.se/'
sci-hub.st: 'https://sci-hub.st/'
sci-hub.ru: 'https://sci-hub.ru/'
default_doi_resolver: 'oadoi.org'
================================================
FILE: searxng/uwsgi.ini
================================================
[uwsgi]
# Who will run the code
uid = searxng
gid = searxng
# Number of workers (usually CPU count)
# default value: %k (= number of CPU core, see Dockerfile)
workers = %k
# Number of threads per worker
# default value: 4 (see Dockerfile)
threads = 4
# The right granted on the created socket
chmod-socket = 666
# Plugin to use and interpreter config
single-interpreter = true
master = true
plugin = python3
lazy-apps = true
enable-threads = 4
# Module to import
module = searx.webapp
# Virtualenv and python path
pythonpath = /usr/local/searxng/
chdir = /usr/local/searxng/searx/
# automatically set processes name to something meaningful
auto-procname = true
# Disable request logging for privacy
disable-logging = true
log-5xx = true
# Set the max size of a request (request-body excluded)
buffer-size = 8192
# No keep alive
# See https://github.com/searx/searx-docker/issues/24
add-header = Connection: close
# uwsgi serves the static files
static-map = /static=/usr/local/searxng/searx/static
# expires set to one day
static-expires = /* 86400
static-gzip-all = True
offload-threads = 4
================================================
FILE: server.py
================================================
# -*- coding: utf-8 -*-
import time
import uvicorn
import sys
import getopt
import json
import os
from pprint import pprint
import requests
import trafilatura
from trafilatura import bare_extraction
from concurrent.futures import ThreadPoolExecutor
import concurrent
import requests
import openai
import time
from datetime import datetime
from urllib.parse import urlparse
import platform
import urllib.parse
import free_ask_internet
from pydantic import BaseModel, Field
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from contextlib import asynccontextmanager
from typing import Any, Dict, List, Literal, Optional, Union
from sse_starlette.sse import ServerSentEvent, EventSourceResponse
from fastapi.responses import StreamingResponse
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
class ModelCard(BaseModel):
id: str
object: str = "model"
created: int = Field(default_factory=lambda: int(time.time()))
owned_by: str = "owner"
root: Optional[str] = None
parent: Optional[str] = None
permission: Optional[list] = None
class ModelList(BaseModel):
object: str = "list"
data: List[ModelCard] = []
class ChatMessage(BaseModel):
role: Literal["user", "assistant", "system"]
content: str
class DeltaMessage(BaseModel):
role: Optional[Literal["user", "assistant", "system"]] = None
content: Optional[str] = None
class QueryRequest(BaseModel):
query:str
model: str
ask_type: Literal["search", "llm"]
llm_auth_token: Optional[str] = "CUSTOM"
llm_base_url: Optional[str] = ""
using_custom_llm:Optional[bool] = False
lang:Optional[str] = "zh-CN"
class ChatCompletionRequest(BaseModel):
model: str
messages: List[ChatMessage]
temperature: Optional[float] = None
top_p: Optional[float] = None
max_length: Optional[int] = None
stream: Optional[bool] = False
class ChatCompletionResponseChoice(BaseModel):
index: int
message: ChatMessage
finish_reason: Literal["stop", "length"]
class ChatCompletionResponseStreamChoice(BaseModel):
index: int
delta: DeltaMessage
finish_reason: Optional[Literal["stop", "length"]]
class ChatCompletionResponse(BaseModel):
model: str
object: Literal["chat.completion", "chat.completion.chunk"]
choices: List[Union[ChatCompletionResponseChoice,
ChatCompletionResponseStreamChoice]]
created: Optional[int] = Field(default_factory=lambda: int(time.time()))
class SearchItem(BaseModel):
url: str
icon_url: str
site_name:str
snippet:str
title:str
class SearchItemList(BaseModel):
search_items: List[SearchItem] = []
class SearchResp(BaseModel):
code:int
msg:str
data: List[SearchItem] = []
@app.get("/v1/models", response_model=ModelList)
async def list_models():
global model_args
model_card = ModelCard(id="gpt-3.5-turbo")
return ModelList(data=[model_card])
@app.post("/v1/chat/completions", response_model=ChatCompletionResponse)
async def create_chat_completion(request: ChatCompletionRequest):
global model, tokenizer
print(request)
if request.messages[-1].role != "user":
raise HTTPException(status_code=400, detail="Invalid request")
query = request.messages[-1].content
generate = predict(query, "", request.model)
return EventSourceResponse(generate, media_type="text/event-stream")
def predict(query: str, history: None, model_id: str):
choice_data = ChatCompletionResponseStreamChoice(
index=0,
delta=DeltaMessage(role="assistant"),
finish_reason=None
)
chunk = ChatCompletionResponse(model=model_id, choices=[
choice_data], object="chat.completion.chunk")
yield "{}".format(chunk.json(exclude_unset=True))
new_response = ""
current_length = 0
for token in free_ask_internet.ask_internet(query=query):
new_response += token
if len(new_response) == current_length:
continue
new_text = new_response[current_length:]
current_length = len(new_response)
choice_data = ChatCompletionResponseStreamChoice(
index=0,
delta=DeltaMessage(content=new_text,role="assistant"),
finish_reason=None
)
chunk = ChatCompletionResponse(model=model_id, choices=[
choice_data], object="chat.completion.chunk")
yield "{}".format(chunk.json(exclude_unset=True))
choice_data = ChatCompletionResponseStreamChoice(
index=0,
delta=DeltaMessage(),
finish_reason="stop"
)
chunk = ChatCompletionResponse(model=model_id, choices=[
choice_data], object="chat.completion.chunk")
yield "{}".format(chunk.json(exclude_unset=True))
yield '[DONE]'
@app.post("/api/search/get_search_refs", response_model=SearchResp)
async def get_search_refs(request: QueryRequest):
global search_results
search_results = []
search_item_list = []
if request.ask_type == "search":
search_links,search_results = free_ask_internet.search_web_ref(request.query)
for search_item in search_links:
snippet = search_item.get("snippet")
url = search_item.get("url")
icon_url = search_item.get("icon_url")
site_name = search_item.get("site_name")
title = search_item.get("title")
si = SearchItem(snippet=snippet,url=url,icon_url=icon_url,site_name=site_name,title=title)
search_item_list.append(si)
resp = SearchResp(code=0,msg="success",data=search_item_list)
return resp
def generator(prompt:str, model:str, llm_auth_token:str,llm_base_url:str, using_custom_llm=False,is_failed=False):
if is_failed:
yield "搜索失败，没有返回结果"
else:
total_token = ""
for token in free_ask_internet.chat(prompt=prompt,model=model,llm_auth_token=llm_auth_token,llm_base_url=llm_base_url,using_custom_llm=using_custom_llm,stream=True):
total_token += token
yield token
@app.post("/api/search/stream/{search_uuid}")
async def stream(search_uuid:str,request: QueryRequest):
global search_results
if request.ask_type == "llm":
answer_language = ' Simplified Chinese '
if request.lang == "zh-CN":
answer_language = ' Simplified Chinese '
if request.lang == "zh-TW":
answer_language = ' Traditional Chinese '
if request.lang == "en-US":
answer_language = ' English '
prompt = ' You are a large language AI assistant develop by nash_su. Answer user question in ' + answer_language + '. And here is the user question: ' + request.query
generate = generator(prompt,model=request.model,llm_auth_token=request.llm_auth_token, llm_base_url=request.llm_base_url, using_custom_llm=request.using_custom_llm)
else:
prompt = None
limit_count = 10
while limit_count > 0:
try:
if len(search_results) > 0:
prompt = free_ask_internet.gen_prompt(request.query,search_results,lang=request.lang,context_length_limit=8000)
break
else:
limit_count -= 1
time.sleep(1)
except Exception as err:
limit_count -= 1
time.sleep(1)
total_token = ""
if prompt:
generate = generator(prompt,model=request.model,llm_auth_token=request.llm_auth_token, llm_base_url=request.llm_base_url, using_custom_llm=request.using_custom_llm)
else:
generate = generator(prompt,model=request.model,llm_auth_token=request.llm_auth_token,llm_base_url=request.llm_base_url, using_custom_llm=request.using_custom_llm,is_failed=True)
# return EventSourceResponse(generate, media_type="text/event-stream")
return StreamingResponse(generate, media_type="text/event-stream")
def main():
port = 8000
search_results = []
uvicorn.run(app, host='0.0.0.0', port=port, workers=1)
if __name__ == "__main__":
main()
gitextract_mcwgdhv6/ ├── .gitignore ├── Dockerfile ├── LICENSE ├── README.md ├── docker-compose.yaml ├── free_ask_internet.py ├── requirements.txt ├── searxng/ │ ├── settings.yml │ └── uwsgi.ini └── server.py
SYMBOL INDEX (24 symbols across 2 files) FILE: free_ask_internet.py function extract_url_content (line 21) | def extract_url_content(url): function search_web_ref (line 30) | def search_web_ref(query:str, debug=False): function gen_prompt (line 102) | def gen_prompt(question,content_list, lang="zh-CN", context_length_limit... function chat (line 171) | def chat(prompt, model:str,llm_auth_token:str,llm_base_url:str,using_cus... function ask_internet (line 217) | def ask_internet(query:str, debug=False): FILE: server.py class ModelCard (line 42) | class ModelCard(BaseModel): class ModelList (line 52) | class ModelList(BaseModel): class ChatMessage (line 57) | class ChatMessage(BaseModel): class DeltaMessage (line 62) | class DeltaMessage(BaseModel): class QueryRequest (line 67) | class QueryRequest(BaseModel): class ChatCompletionRequest (line 76) | class ChatCompletionRequest(BaseModel): class ChatCompletionResponseChoice (line 85) | class ChatCompletionResponseChoice(BaseModel): class ChatCompletionResponseStreamChoice (line 91) | class ChatCompletionResponseStreamChoice(BaseModel): class ChatCompletionResponse (line 97) | class ChatCompletionResponse(BaseModel): class SearchItem (line 104) | class SearchItem(BaseModel): class SearchItemList (line 111) | class SearchItemList(BaseModel): class SearchResp (line 114) | class SearchResp(BaseModel): function list_models (line 121) | async def list_models(): function create_chat_completion (line 128) | async def create_chat_completion(request: ChatCompletionRequest): function predict (line 141) | def predict(query: str, history: None, model_id: str): function get_search_refs (line 183) | async def get_search_refs(request: QueryRequest): function generator (line 206) | def generator(prompt:str, model:str, llm_auth_token:str,llm_base_url:str... function stream (line 216) | async def stream(search_uuid:str,request: QueryRequest): function main (line 254) | def main():
Condensed preview — 10 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (102K chars).
[
{
"path": ".gitignore",
"chars": 35,
"preview": "__pycache__/\n*.py[cod]\n*$py.class\n\n"
},
{
"path": "Dockerfile",
"chars": 259,
"preview": "FROM python:3.9.15\n\nWORKDIR /app\nCOPY requirements.txt /app\nRUN pip3 install -i https://mirrors.aliyun.com/pypi/simple/"
},
{
"path": "LICENSE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 4497,
"preview": "# FreeAskInternet\n\n## 🎉🎉🎉 Yeah we have a logo now! 🎉🎉🎉\n\n\n\n> Running www.perplexity.ai li"
},
{
"path": "docker-compose.yaml",
"chars": 1491,
"preview": "services:\n backend:\n image: docker.io/nashsu/free_ask_internet:latest\n depends_on:\n - llm-freegpt35\n rest"
},
{
"path": "free_ask_internet.py",
"chars": 8799,
"preview": "# -*- coding: utf-8 -*-\n\nimport json\nimport os \nfrom pprint import pprint\nimport requests\nimport trafilatura\nfrom trafil"
},
{
"path": "requirements.txt",
"chars": 655,
"preview": "annotated-types==0.6.0\nanyio==4.3.0\ncertifi==2024.2.2\ncharset-normalizer==3.3.2\nclick==8.1.7\ncourlan==1.0.0\ndateparser=="
},
{
"path": "searxng/settings.yml",
"chars": 61408,
"preview": "general:\n # Debug mode, only for development. Is overwritten by ${SEARXNG_DEBUG}\n debug: false\n # displayed name\n in"
},
{
"path": "searxng/uwsgi.ini",
"chars": 1103,
"preview": "[uwsgi]\n# Who will run the code\nuid = searxng\ngid = searxng\n\n# Number of workers (usually CPU count)\n# default value: %k"
},
{
"path": "server.py",
"chars": 8345,
"preview": "# -*- coding: utf-8 -*-\n\nimport time\nimport uvicorn\nimport sys\nimport getopt\nimport json\nimport os \nfrom pprint import p"
}
]
About this extraction
This page contains the full source code of the nashsu/FreeAskInternet GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 10 files (95.7 KB), approximately 27.0k tokens, and a symbol index with 24 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.