Showing preview only (361K chars total). Download the full file or copy to clipboard to get everything.
Repository: ggreer/the_silver_searcher
Branch: master
Commit: a61f1780b642
Files: 95
Total size: 338.7 KB
Directory structure:
gitextract_qvo96f4r/
├── .clang-format
├── .gitignore
├── .travis.yml
├── CONTRIBUTING.md
├── LICENSE
├── Makefile.am
├── Makefile.w32
├── NOTICE
├── README.md
├── _the_silver_searcher
├── ag.bashcomp.sh
├── autogen.sh
├── build.sh
├── configure.ac
├── doc/
│ ├── ag.1
│ ├── ag.1.md
│ └── generate_man.sh
├── format.sh
├── m4/
│ └── ax_pthread.m4
├── pgo.sh
├── sanitize.sh
├── src/
│ ├── decompress.c
│ ├── decompress.h
│ ├── ignore.c
│ ├── ignore.h
│ ├── lang.c
│ ├── lang.h
│ ├── log.c
│ ├── log.h
│ ├── main.c
│ ├── options.c
│ ├── options.h
│ ├── print.c
│ ├── print.h
│ ├── print_w32.c
│ ├── scandir.c
│ ├── scandir.h
│ ├── search.c
│ ├── search.h
│ ├── uthash.h
│ ├── util.c
│ ├── util.h
│ ├── win32/
│ │ └── config.h
│ └── zfile.c
├── tests/
│ ├── adjacent_matches.t
│ ├── bad_path.t
│ ├── big/
│ │ ├── big_file.t
│ │ └── create_big_file.py
│ ├── case_sensitivity.t
│ ├── color.t
│ ├── column.t
│ ├── count.t
│ ├── ds_store_ignore.t
│ ├── empty_environment.t
│ ├── empty_match.t
│ ├── exitcodes.t
│ ├── fail/
│ │ ├── unicode_case_insensitive.t
│ │ └── unicode_case_insensitive.t.err
│ ├── files_with_matches.t
│ ├── filetype.t
│ ├── hidden_option.t
│ ├── ignore_abs_path.t
│ ├── ignore_absolute_search_path_with_glob.t
│ ├── ignore_backups.t
│ ├── ignore_examine_parent_ignorefiles.t
│ ├── ignore_extensions.t
│ ├── ignore_gitignore.t
│ ├── ignore_invert.t
│ ├── ignore_pattern_in_subdirectory.t
│ ├── ignore_slash_in_subdir.t
│ ├── ignore_subdir.t
│ ├── ignore_vcs.t
│ ├── invert_match.t
│ ├── is_binary_pdf.t
│ ├── line_width.t
│ ├── list_file_types.t
│ ├── literal_word_regexp.t
│ ├── max_count.t
│ ├── multiline.t
│ ├── negated_options.t
│ ├── one_device.t
│ ├── only_matching.t
│ ├── option_g.t
│ ├── option_smartcase.t
│ ├── passthrough.t
│ ├── pipecontext.t
│ ├── print_all_files.t
│ ├── print_end.t
│ ├── print_end.txt
│ ├── search_stdin.t
│ ├── setup.sh
│ ├── stupid_fnmatch.t.disabled
│ ├── vimgrep.t
│ └── word_regexp.t
└── the_silver_searcher.spec.in
================================================
FILE CONTENTS
================================================
================================================
FILE: .clang-format
================================================
AlignEscapedNewlinesLeft: false
AlignTrailingComments: true
AllowAllParametersOfDeclarationOnNextLine: true
AllowShortBlocksOnASingleLine: false
AllowShortCaseLabelsOnASingleLine: false
AllowShortFunctionsOnASingleLine: None
AllowShortIfStatementsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AlwaysBreakAfterDefinitionReturnType: false
AlwaysBreakBeforeMultilineStrings: false
BinPackArguments: true
BinPackParameters: true
BreakBeforeBinaryOperators: None
BreakBeforeBraces: Attach
BreakBeforeTernaryOperators: true
ColumnLimit: 0
ContinuationIndentWidth: 4
Cpp11BracedListStyle: false
DerivePointerAlignment: false
DisableFormat: false
ExperimentalAutoDetectBinPacking: false
IndentCaseLabels: true
IndentFunctionDeclarationAfterType: false
IndentWidth: 4
IndentWrappedFunctionNames: false
KeepEmptyLinesAtTheStartOfBlocks: false
Language: Cpp
MaxEmptyLinesToKeep: 2
PointerAlignment: Right
SpaceAfterCStyleCast: false
SpaceBeforeAssignmentOperators: true
SpaceBeforeParens: ControlStatements
SpaceInEmptyParentheses: false
SpacesBeforeTrailingComments: 1
SpacesInCStyleCastParentheses: false
SpacesInParentheses: false
SpacesInSquareBrackets: false
UseTab: Never
================================================
FILE: .gitignore
================================================
*.dSYM
*.gcda
*.o
*.plist
.deps
.dirstamp
.DS_Store
aclocal.m4
ag
ag.exe
autom4te.cache
cachegrind.out.*
callgrind.out.*
clang_output_*
compile
config.guess
config.log
config.status
config.sub
configure
depcomp
gmon.out
install-sh
Makefile
Makefile.in
missing
src/config.h*
stamp-h1
tests/*.err
tests/big/*.err
tests/big/big_file.txt
the_silver_searcher.spec
================================================
FILE: .travis.yml
================================================
language: c
dist: xenial
sudo: false
branches:
only:
- master
- ppc64le
arch:
- amd64
- ppc64le
compiler:
- clang
- gcc
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- automake
- liblzma-dev
- libpcre3-dev
- pkg-config
- zlib1g-dev
env:
global:
- LLVM_VERSION=6.0.1
- LLVM_PATH=$HOME/clang+llvm
- CLANG_FORMAT=$LLVM_PATH/bin/clang-format
before_install:
- wget http://llvm.org/releases/$LLVM_VERSION/clang+llvm-$LLVM_VERSION-x86_64-linux-gnu-ubuntu-16.04.tar.xz -O $LLVM_PATH.tar.xz
- mkdir $LLVM_PATH
- tar xf $LLVM_PATH.tar.xz -C $LLVM_PATH --strip-components=1
- export PATH=$HOME/.local/bin:$PATH
install:
- pip install --user cram
script:
- ./build.sh && make test
notifications:
irc: 'chat.freenode.net#ag'
on_success: change
on_failure: always
use_notice: true
================================================
FILE: CONTRIBUTING.md
================================================
## Contributing
I like when people send pull requests. It validates my existence. If you want to help out, check the [issue list](https://github.com/ggreer/the_silver_searcher/issues?sort=updated&state=open) or search the codebase for `TODO`. Don't worry if you lack experience writing C. If I think a pull request isn't ready to be merged, I'll give feedback in comments. Once everything looks good, I'll comment on your pull request with a cool animated gif and hit the merge button.
### Running the test suite
If you contribute, you might want to run the test suite before and after writing
some code, just to make sure you did not break anything. Adding tests along with
your code is nice to have, because it makes regressions less likely to happen.
Also, if you think you have found a bug, contributing a failing test case is a
good way of making your point and adding value at the same time.
The test suite uses [Cram](https://bitheap.org/cram/). You'll need to build ag
first, and then you can run the suite from the root of the repository :
make test
### Adding filetypes
Ag can search files which belong to a certain class for example `ag --html test`
searches all files with the extension defined in [lang.c](src/lang.c).
If you want to add a new file 'class' to ag please modify [lang.c](src/lang.c) and [list_file_types.t](tests/list_file_types.t).
`lang.c` adds the functionality and `list_file_types.t` adds the test case.
Without adding a test case the test __will__ fail.
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: Makefile.am
================================================
ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS}
bin_PROGRAMS = ag
ag_SOURCES = src/ignore.c src/ignore.h src/log.c src/log.h src/options.c src/options.h src/print.c src/print_w32.c src/print.h src/scandir.c src/scandir.h src/search.c src/search.h src/lang.c src/lang.h src/util.c src/util.h src/decompress.c src/decompress.h src/uthash.h src/main.c src/zfile.c
ag_LDADD = ${PCRE_LIBS} ${LZMA_LIBS} ${ZLIB_LIBS} $(PTHREAD_LIBS)
dist_man_MANS = doc/ag.1
bashcompdir = $(pkgdatadir)/completions
dist_bashcomp_DATA = ag.bashcomp.sh
zshcompdir = $(datadir)/zsh/site-functions
dist_zshcomp_DATA = _the_silver_searcher
EXTRA_DIST = Makefile.w32 LICENSE NOTICE the_silver_searcher.spec README.md
all:
@$(MAKE) ag -r
test: ag
cram -v tests/*.t
if HAS_CLANG_FORMAT
CLANG_FORMAT=${CLANG_FORMAT} ./format.sh test
else
@echo "clang-format is not available. Skipped clang-format test."
endif
test_big: ag
cram -v tests/big/*.t
test_fail: ag
cram -v tests/fail/*.t
.PHONY : all clean test test_big test_fail
================================================
FILE: Makefile.w32
================================================
SED=sed
VERSION:=$(shell "$(SED)" -n "s/[^[]*\[\([0-9]\+\.[0-9]\+\.[0-9]\+\)\],/\1/p" configure.ac)
CC=gcc
RM=/bin/rm
SRCS = \
src/decompress.c \
src/ignore.c \
src/lang.c \
src/log.c \
src/main.c \
src/options.c \
src/print.c \
src/scandir.c \
src/search.c \
src/util.c \
src/print_w32.c
OBJS = $(subst .c,.o,$(SRCS))
CFLAGS = -O2 -Isrc/win32 -DPACKAGE_VERSION=\"$(VERSION)\"
LIBS = -lz -lpthread -lpcre -llzma -lshlwapi
TARGET = ag.exe
all : $(TARGET)
# depend on configure.ac to account for version changes
$(TARGET) : $(OBJS) configure.ac
$(CC) -o $@ $(OBJS) $(LIBS)
.c.o :
$(CC) -c $(CFLAGS) -Isrc $< -o $@
clean :
$(RM) -f src/*.o $(TARGET)
================================================
FILE: NOTICE
================================================
The Silver Searcher
Copyright 2011-2016 Geoff Greer
================================================
FILE: README.md
================================================
# The Silver Searcher
A code searching tool similar to `ack`, with a focus on speed.
[](https://travis-ci.org/ggreer/the_silver_searcher)
[](https://floobits.com/ggreer/ag/redirect)
[](https://webchat.freenode.net/?channels=ag)
Do you know C? Want to improve ag? [I invite you to pair with me](http://geoff.greer.fm/2014/10/13/help-me-get-to-ag-10/).
## What's so great about Ag?
* It is an order of magnitude faster than `ack`.
* It ignores file patterns from your `.gitignore` and `.hgignore`.
* If there are files in your source repo you don't want to search, just add their patterns to a `.ignore` file. (\*cough\* `*.min.js` \*cough\*)
* The command name is 33% shorter than `ack`, and all keys are on the home row!
Ag is quite stable now. Most changes are new features, minor bug fixes, or performance improvements. It's much faster than Ack in my benchmarks:
ack test_blah ~/code/ 104.66s user 4.82s system 99% cpu 1:50.03 total
ag test_blah ~/code/ 4.67s user 4.58s system 286% cpu 3.227 total
Ack and Ag found the same results, but Ag was 34x faster (3.2 seconds vs 110 seconds). My `~/code` directory is about 8GB. Thanks to git/hg/ignore, Ag only searched 700MB of that.
There are also [graphs of performance across releases](http://geoff.greer.fm/ag/speed/).
## How is it so fast?
* Ag uses [Pthreads](https://en.wikipedia.org/wiki/POSIX_Threads) to take advantage of multiple CPU cores and search files in parallel.
* Files are `mmap()`ed instead of read into a buffer.
* Literal string searching uses [Boyer-Moore strstr](https://en.wikipedia.org/wiki/Boyer%E2%80%93Moore_string_search_algorithm).
* Regex searching uses [PCRE's JIT compiler](http://sljit.sourceforge.net/pcre.html) (if Ag is built with PCRE >=8.21).
* Ag calls `pcre_study()` before executing the same regex on every file.
* Instead of calling `fnmatch()` on every pattern in your ignore files, non-regex patterns are loaded into arrays and binary searched.
I've written several blog posts showing how I've improved performance. These include how I [added pthreads](http://geoff.greer.fm/2012/09/07/the-silver-searcher-adding-pthreads/), [wrote my own `scandir()`](http://geoff.greer.fm/2012/09/03/profiling-ag-writing-my-own-scandir/), [benchmarked every revision to find performance regressions](http://geoff.greer.fm/2012/08/25/the-silver-searcher-benchmarking-revisions/), and profiled with [gprof](http://geoff.greer.fm/2012/02/08/profiling-with-gprof/) and [Valgrind](http://geoff.greer.fm/2012/01/23/making-programs-faster-profiling/).
## Installing
### macOS
brew install the_silver_searcher
or
port install the_silver_searcher
### Linux
* Ubuntu >= 13.10 (Saucy) or Debian >= 8 (Jessie)
apt-get install silversearcher-ag
* Fedora 21 and lower
yum install the_silver_searcher
* Fedora 22+
dnf install the_silver_searcher
* RHEL7+
yum install epel-release.noarch the_silver_searcher
* Gentoo
emerge -a sys-apps/the_silver_searcher
* Arch
pacman -S the_silver_searcher
* Slackware
sbopkg -i the_silver_searcher
* openSUSE
zypper install the_silver_searcher
* CentOS
yum install the_silver_searcher
* NixOS/Nix/Nixpkgs
nix-env -iA silver-searcher
* SUSE Linux Enterprise: Follow [these simple instructions](https://software.opensuse.org/download.html?project=utilities&package=the_silver_searcher).
### BSD
* FreeBSD
pkg install the_silver_searcher
* OpenBSD/NetBSD
pkg_add the_silver_searcher
### Windows
* Win32/64
Unofficial daily builds are [available](https://github.com/k-takata/the_silver_searcher-win32).
* winget
winget install "The Silver Searcher"
Notes:
- This installs a [release](https://github.com/JFLarvoire/the_silver_searcher/releases) of ag.exe optimized for Windows.
- winget is intended to become the default package manager client for Windows.
As of June 2020, it's still in beta, and can be installed using instructions [there](https://github.com/microsoft/winget-cli).
- The setup script in the Ag's winget package installs ag.exe in the first directory that matches one of these criteria:
1. Over a previous instance of ag.exe *from the same [origin](https://github.com/JFLarvoire/the_silver_searcher)* found in the PATH
2. In the directory defined in environment variable bindir_%PROCESSOR_ARCHITECTURE%
3. In the directory defined in environment variable bindir
4. In the directory defined in environment variable windir
* Chocolatey
choco install ag
* MSYS2
pacman -S mingw-w64-{i686,x86_64}-ag
* Cygwin
Run the relevant [`setup-*.exe`](https://cygwin.com/install.html), and select "the\_silver\_searcher" in the "Utils" category.
## Building from source
### Building master
1. Install dependencies (Automake, pkg-config, PCRE, LZMA):
* macOS:
brew install automake pkg-config pcre xz
or
port install automake pkgconfig pcre xz
* Ubuntu/Debian:
apt-get install -y automake pkg-config libpcre3-dev zlib1g-dev liblzma-dev
* Fedora:
yum -y install pkgconfig automake gcc zlib-devel pcre-devel xz-devel
* CentOS:
yum -y groupinstall "Development Tools"
yum -y install pcre-devel xz-devel zlib-devel
* openSUSE:
zypper source-install --build-deps-only the_silver_searcher
* Windows: It's complicated. See [this wiki page](https://github.com/ggreer/the_silver_searcher/wiki/Windows).
2. Run the build script (which just runs aclocal, automake, etc):
./build.sh
On Windows (inside an msys/MinGW shell):
make -f Makefile.w32
3. Make install:
sudo make install
### Building a release tarball
GPG-signed releases are available [here](http://geoff.greer.fm/ag).
Building release tarballs requires the same dependencies, except for automake and pkg-config. Once you've installed the dependencies, just run:
./configure
make
make install
You may need to use `sudo` or run as root for the make install.
## Editor Integration
### Vim
You can use Ag with [ack.vim](https://github.com/mileszs/ack.vim) by adding the following line to your `.vimrc`:
let g:ackprg = 'ag --nogroup --nocolor --column'
or:
let g:ackprg = 'ag --vimgrep'
Which has the same effect but will report every match on the line.
### Emacs
You can use [ag.el][] as an Emacs front-end to Ag. See also: [helm-ag].
[ag.el]: https://github.com/Wilfred/ag.el
[helm-ag]: https://github.com/syohex/emacs-helm-ag
### TextMate
TextMate users can use Ag with [my fork](https://github.com/ggreer/AckMate) of the popular AckMate plugin, which lets you use both Ack and Ag for searching. If you already have AckMate you just want to replace Ack with Ag, move or delete `"~/Library/Application Support/TextMate/PlugIns/AckMate.tmplugin/Contents/Resources/ackmate_ack"` and run `ln -s /usr/local/bin/ag "~/Library/Application Support/TextMate/PlugIns/AckMate.tmplugin/Contents/Resources/ackmate_ack"`
## Other stuff you might like
* [Ack](https://github.com/petdance/ack3) - Better than grep. Without Ack, Ag would not exist.
* [ack.vim](https://github.com/mileszs/ack.vim)
* [Exuberant Ctags](http://ctags.sourceforge.net/) - Faster than Ag, but it builds an index beforehand. Good for *really* big codebases.
* [Git-grep](http://git-scm.com/docs/git-grep) - As fast as Ag but only works on git repos.
* [fzf](https://github.com/junegunn/fzf) - A command-line fuzzy finder
* [ripgrep](https://github.com/BurntSushi/ripgrep)
* [Sack](https://github.com/sampson-chen/sack) - A utility that wraps Ack and Ag. It removes a lot of repetition from searching and opening matching files.
================================================
FILE: _the_silver_searcher
================================================
#compdef ag
# Completion function for zsh
local ret=1
local -a args expl
# Intentionally avoided many possible mutual exlusions because it is
# likely that earlier options come from an alias. In line with this
# the following conditionally adds options that assert defaults.
[[ -n $words[(r)(-[is]|--ignore-case|--case-sensitive)] ]] && args+=(
'(-S --smart-case -s -s --ignore-case --case-sensitive)'{-S,--smart-case}'[insensitive match unless pattern includes uppercase]'
)
[[ -n $words[(r)--nobreak] ]] && args+=(
"(--nobreak)--break[print newlines between matches in different files]"
)
[[ -n $words[(r)--nogroup] ]] && args+=(
"(--nogroup)--group[don't repeat filename for each match line]"
)
_tags normal-options file-types
while _tags; do
_requested normal-options && _arguments -S -s $args \
'--ackmate[print results in AckMate-parseable format]' \
'(--after -A)'{--after=-,-A+}'[specify lines of trailing context]::lines [2]' \
'(--before -B)'{--before=-,-B+}'[specify lines of leading context]::lines [2]' \
"--nobreak[don't print newlines between matches in different files]" \
'(--count -c)'{--count,-c}'[only print a count of matching lines]' \
'--color[enable color highlighting of output]' \
'(--color-line-number --color-match --color-path)--nocolor[disable color highlighting of output]' \
'--color-line-number=[specify color for line numbers]:color [1;33]' \
'--color-match=[specify color for result match numbers]:color [30;43]' \
'--color-path=[specify color for path names]:color [1;32]' \
'--column[print column numbers in results]' \
'(--context -C)'{--context=-,-C+}'[specify lines of context]::lines' \
'(--debug -D)'{--debug,-D}'[output debug information]' \
'--depth=[specify directory levels to descend when searching]:levels [25]' \
'(--noheading)--nofilename[suppress printing of filenames]' \
'(-f --follow)'{-f,--follow}'[follow symlinks]' \
'(-F --fixed-strings --literal -Q)'{--fixed-strings,-F,--literal,-Q}'[use literal strings]' \
'--nogroup[repeat filename for each match line]' \
'(1 -G --file-search-regex)-g+[print filenames matching a pattern]:regex' \
'(-G --file-search-regex)'{-G+,--file-search-regex=}'[limit search to filenames matching pattern]:regex' \
'(-H --heading --noheading)'{-H,--heading}'[print filename with each match]' \
'(-H --heading --noheading --nofilename)--noheading[suppress printing of filenames]' \
'--hidden[search hidden files (obeying .*ignore files)]' \
{--ignore=,--ignore-dir=}'[ignore files/directories matching pattern]:regex' \
'(-i --ignore-case)'{-i,--ignore-case}'[match case-insensitively]' \
'(-l --files-with-matches)'{-l,--files-with-matches}"[output matching files' names only]" \
'(-L --files-without-matches)'{-L,--files-without-matches}"[output non-matching files' names only]" \
'(--max-count -m)'{--max-count=,-m+}'[stop after specified no of matches in each file]:max number of matches' \
'--numbers[prefix output with line numbers, even for streams]' \
'--nonumbers[suppress printing of line numbers]' \
'(--only-matching -o)'{--only-matching,-o}'[show only matching part of line]' \
'(-p --path-to-ignore)'{-p+,--path-to-ignore=}'[use specified .ignore file]:file:_files' \
'--print-long-lines[print matches on very long lines]' \
"--passthrough[when searching a stream, print all lines even if they don't match]" \
'(-s --case-sensitive)'{-s,--case-sensitive}'[match case]' \
'--silent[suppress all log messages, including errors]' \
'(--stats-only)--stats[print stats (files scanned, time taken, etc.)]' \
'(--stats)--stats-only[print stats and nothing else]' \
'(-U --skip-vcs-ignores)'{-U,--skip-vcs-ignores}'[ignore VCS files (stil obey .ignore)]' \
'(-v --invert-match)'{-v,--invert-match}'[select non-matching lines]' \
'--vimgrep[output results like vim :vimgrep /pattern/g would]' \
'(-w --word-regexp)'{-w,--word-regexp}'[force pattern to match only whole words]' \
'(-z --search-zip)'{-z,--search-zip}'[search contents of compressed files]' \
'(-0 --null)'{-0,--null}'[separate filenames with null]' \
': :_guard "^-*" pattern' \
'*:file:_files' \
'(- :)--list-file-types[list supported file types]' \
'(- :)'{-h,--help}'[display help information]' \
'(- :)'{-V,--version}'[display version information]' \
- '(ignores)' \
'(-a --all-types)'{-a,--all-types}'[search all files]' \
'--search-binary[search binary files for matches]' \
{-t,--all-text}'[search all text files (not including hidden files)]' \
{-u,--unrestricted}'[search all files]' && ret=0
_requested file-types && { ! zstyle -T ":completion:${curcontext}:options" prefix-needed ||
[[ -prefix - ]] } && _all_labels file-types expl 'file type' \
compadd - ${(M)$(_call_program file-types $words[1] --list-file-types):#--*} && ret=0
(( ret )) || break
done
return ret
================================================
FILE: ag.bashcomp.sh
================================================
_ag() {
local lngopt shtopt split=false
local cur prev
COMPREPLY=()
cur=$(_get_cword "=")
prev="${COMP_WORDS[COMP_CWORD-1]}"
_expand || return 0
lngopt='
--ackmate
--ackmate-dir-filter
--affinity
--after
--all-text
--all-types
--before
--break
--case-sensitive
--color
--color-line-number
--color-match
--color-path
--color-win-ansi
--column
--context
--count
--debug
--depth
--file-search-regex
--filename
--files-with-matches
--files-without-matches
--fixed-strings
--follow
--group
--heading
--help
--hidden
--ignore
--ignore-case
--ignore-dir
--invert-match
--line-numbers
--list-file-types
--literal
--match
--max-count
--no-numbers
--no-recurse
--noaffinity
--nobreak
--nocolor
--nofilename
--nofollow
--nogroup
--noheading
--nonumbers
--nopager
--norecurse
--null
--numbers
--one-device
--only-matching
--pager
--parallel
--passthrough
--passthru
--path-to-ignore
--print-long-lines
--print0
--recurse
--search-binary
--search-files
--search-zip
--silent
--skip-vcs-ignores
--smart-case
--stats
--unrestricted
--version
--vimgrep
--word-regexp
--workers
'
shtopt='
-a -A -B -C -D
-f -F -g -G -h
-i -l -L -m -n
-p -Q -r -R -s
-S -t -u -U -v
-V -w -z
'
types=$(ag --list-file-types |grep -- '--')
# these options require an argument
if [[ "${prev}" == -[ABCGgm] ]] ; then
return 0
fi
_split_longopt && split=true
case "${prev}" in
--ignore-dir) # directory completion
_filedir -d
return 0;;
--path-to-ignore) # file completion
_filedir
return 0;;
--pager) # command completion
COMPREPLY=( $(compgen -c -- "${cur}") )
return 0;;
--ackmate-dir-filter|--after|--before|--color-*|--context|--depth\
|--file-search-regex|--ignore|--max-count|--workers)
return 0;;
esac
$split && return 0
case "${cur}" in
-*)
COMPREPLY=( $(compgen -W \
"${lngopt} ${shtopt} ${types}" -- "${cur}") )
return 0;;
*)
_filedir
return 0;;
esac
} &&
# shellcheck disable=SC2086
# shellcheck disable=SC2154,SC2086
complete -F _ag ${nospace} ag
================================================
FILE: autogen.sh
================================================
#!/bin/sh
set -e
cd "$(dirname "$0")"
AC_SEARCH_OPTS=""
# For those of us with pkg-config and other tools in /usr/local
PATH=$PATH:/usr/local/bin
# This is to make life easier for people who installed pkg-config in /usr/local
# but have autoconf/make/etc in /usr/. AKA most mac users
if [ -d "/usr/local/share/aclocal" ]
then
AC_SEARCH_OPTS="-I /usr/local/share/aclocal"
fi
# shellcheck disable=2086
aclocal $AC_SEARCH_OPTS
autoconf
autoheader
automake --add-missing
================================================
FILE: build.sh
================================================
#!/bin/sh
set -e
cd "$(dirname "$0")"
./autogen.sh
./configure "$@"
make -j4
================================================
FILE: configure.ac
================================================
AC_INIT(
[the_silver_searcher],
[2.2.0],
[https://github.com/ggreer/the_silver_searcher/issues],
[the_silver_searcher],
[https://github.com/ggreer/the_silver_searcher])
AM_INIT_AUTOMAKE([no-define foreign subdir-objects])
AC_PROG_CC
AM_PROG_CC_C_O
AC_PREREQ([2.59])
AC_PROG_GREP
m4_ifdef(
[AM_SILENT_RULES],
[AM_SILENT_RULES([yes])])
PKG_CHECK_MODULES([PCRE], [libpcre])
m4_include([m4/ax_pthread.m4])
AX_PTHREAD(
[AC_CHECK_HEADERS([pthread.h])],
[AC_MSG_WARN([No pthread support. Ag will be slower due to running single-threaded.])]
)
# Run CFLAGS="-pg" ./configure if you want debug symbols
if ! echo "$CFLAGS" | "$GREP" '\(^\|[[[:space:]]]\)-O' > /dev/null; then
CFLAGS="$CFLAGS -O2"
fi
CFLAGS="$CFLAGS $PTHREAD_CFLAGS $PCRE_CFLAGS -Wall -Wextra -Wformat=2 -Wno-format-nonliteral -Wshadow"
CFLAGS="$CFLAGS -Wpointer-arith -Wcast-qual -Wmissing-prototypes -Wno-missing-braces -std=gnu89 -D_GNU_SOURCE"
LDFLAGS="$LDFLAGS"
case $host in
*mingw*)
AC_CHECK_LIB(shlwapi, main,, AC_MSG_ERROR(libshlwapi missing))
esac
LIBS="$PTHREAD_LIBS $LIBS"
AC_ARG_ENABLE([zlib],
AS_HELP_STRING([--disable-zlib], [Disable zlib compressed search support]))
AS_IF([test "x$enable_zlib" != "xno"], [
AC_CHECK_HEADERS([zlib.h])
AC_SEARCH_LIBS([inflate], [zlib, z])
])
AC_ARG_ENABLE([lzma],
AS_HELP_STRING([--disable-lzma], [Disable lzma compressed search support]))
AS_IF([test "x$enable_lzma" != "xno"], [
AC_CHECK_HEADERS([lzma.h])
PKG_CHECK_MODULES([LZMA], [liblzma])
])
AC_CHECK_DECL([PCRE_CONFIG_JIT], [AC_DEFINE([USE_PCRE_JIT], [], [Use PCRE JIT])], [], [#include <pcre.h>])
AC_CHECK_DECL([CPU_ZERO, CPU_SET], [AC_DEFINE([USE_CPU_SET], [], [Use CPU_SET macros])] , [], [#include <sched.h>])
AC_CHECK_HEADERS([sys/cpuset.h err.h])
AC_CHECK_MEMBER([struct dirent.d_type], [AC_DEFINE([HAVE_DIRENT_DTYPE], [], [Have dirent struct member d_type])], [], [[#include <dirent.h>]])
AC_CHECK_MEMBER([struct dirent.d_namlen], [AC_DEFINE([HAVE_DIRENT_DNAMLEN], [], [Have dirent struct member d_namlen])], [], [[#include <dirent.h>]])
AC_CHECK_FUNCS(fgetln fopencookie getline realpath strlcpy strndup vasprintf madvise posix_fadvise pthread_setaffinity_np pledge)
AC_CONFIG_FILES([Makefile the_silver_searcher.spec])
AC_CONFIG_HEADERS([src/config.h])
AC_CHECK_PROGS(
[CLANG_FORMAT],
[clang-format-3.8 clang-format-3.7 clang-format-3.6 clang-format],
[no]
)
AM_CONDITIONAL([HAS_CLANG_FORMAT], [test x$CLANG_FORMAT != xno])
AM_COND_IF(
[HAS_CLANG_FORMAT],
[AC_MSG_NOTICE([clang-format found. 'make test' will detect improperly-formatted files.])],
[AC_MSG_WARN([clang-format not found. 'make test' will not detect improperly-formatted files.])]
)
AC_OUTPUT
================================================
FILE: doc/ag.1
================================================
.\" generated with Ronn/v0.7.3
.\" http://github.com/rtomayko/ronn/tree/0.7.3
.
.TH "AG" "1" "December 2016" "" ""
.
.SH "NAME"
\fBag\fR \- The Silver Searcher\. Like ack, but faster\.
.
.SH "SYNOPSIS"
\fBag\fR [\fIoptions\fR] \fIpattern\fR [\fIpath \.\.\.\fR]
.
.SH "DESCRIPTION"
Recursively search for PATTERN in PATH\. Like grep or ack, but faster\.
.
.SH "OPTIONS"
.
.TP
\fB\-\-ackmate\fR
Output results in a format parseable by AckMate \fIhttps://github\.com/protocool/AckMate\fR\.
.
.TP
\fB\-\-[no]affinity\fR
Set thread affinity (if platform supports it)\. Default is true\.
.
.TP
\fB\-a \-\-all\-types\fR
Search all files\. This doesn\'t include hidden files, and doesn\'t respect any ignore files\.
.
.TP
\fB\-A \-\-after [LINES]\fR
Print lines after match\. If not provided, LINES defaults to 2\.
.
.TP
\fB\-B \-\-before [LINES]\fR
Print lines before match\. If not provided, LINES defaults to 2\.
.
.TP
\fB\-\-[no]break\fR
Print a newline between matches in different files\. Enabled by default\.
.
.TP
\fB\-c \-\-count\fR
Only print the number of matches in each file\. Note: This is the number of matches, \fBnot\fR the number of matching lines\. Pipe output to \fBwc \-l\fR if you want the number of matching lines\.
.
.TP
\fB\-\-[no]color\fR
Print color codes in results\. Enabled by default\.
.
.TP
\fB\-\-color\-line\-number\fR
Color codes for line numbers\. Default is 1;33\.
.
.TP
\fB\-\-color\-match\fR
Color codes for result match numbers\. Default is 30;43\.
.
.TP
\fB\-\-color\-path\fR
Color codes for path names\. Default is 1;32\.
.
.TP
\fB\-\-column\fR
Print column numbers in results\.
.
.TP
\fB\-C \-\-context [LINES]\fR
Print lines before and after matches\. Default is 2\.
.
.TP
\fB\-D \-\-debug\fR
Output ridiculous amounts of debugging info\. Not useful unless you\'re actually debugging\.
.
.TP
\fB\-\-depth NUM\fR
Search up to NUM directories deep, \-1 for unlimited\. Default is 25\.
.
.TP
\fB\-\-[no]filename\fR
Print file names\. Enabled by default, except when searching a single file\.
.
.TP
\fB\-f \-\-[no]follow\fR
Follow symlinks\. Default is false\.
.
.TP
\fB\-F \-\-fixed\-strings\fR
Alias for \-\-literal for compatibility with grep\.
.
.TP
\fB\-\-[no]group\fR
The default, \fB\-\-group\fR, lumps multiple matches in the same file together, and presents them under a single occurrence of the filename\. \fB\-\-nogroup\fR refrains from this, and instead places the filename at the start of each match line\.
.
.TP
\fB\-g PATTERN\fR
Print filenames matching PATTERN\.
.
.TP
\fB\-G \-\-file\-search\-regex PATTERN\fR
Only search files whose names match PATTERN\.
.
.TP
\fB\-H \-\-[no]heading\fR
Print filenames above matching contents\.
.
.TP
\fB\-\-hidden\fR
Search hidden files\. This option obeys ignored files\.
.
.TP
\fB\-\-ignore PATTERN\fR
Ignore files/directories whose names match this pattern\. Literal file and directory names are also allowed\.
.
.TP
\fB\-\-ignore\-dir NAME\fR
Alias for \-\-ignore for compatibility with ack\.
.
.TP
\fB\-i \-\-ignore\-case\fR
Match case\-insensitively\.
.
.TP
\fB\-l \-\-files\-with\-matches\fR
Only print the names of files containing matches, not the matching lines\. An empty query will print all files that would be searched\.
.
.TP
\fB\-L \-\-files\-without\-matches\fR
Only print the names of files that don\'t contain matches\.
.
.TP
\fB\-\-list\-file\-types\fR
See \fBFILE TYPES\fR below\.
.
.TP
\fB\-m \-\-max\-count NUM\fR
Skip the rest of a file after NUM matches\. Default is 0, which never skips\.
.
.TP
\fB\-\-[no]mmap\fR
Toggle use of memory\-mapped I/O\. Defaults to true on platforms where \fBmmap()\fR is faster than \fBread()\fR\. (All but macOS\.)
.
.TP
\fB\-\-[no]multiline\fR
Match regexes across newlines\. Enabled by default\.
.
.TP
\fB\-n \-\-norecurse\fR
Don\'t recurse into directories\.
.
.TP
\fB\-\-[no]numbers\fR
Print line numbers\. Default is to omit line numbers when searching streams\.
.
.TP
\fB\-o \-\-only\-matching\fR
Print only the matching part of the lines\.
.
.TP
\fB\-\-one\-device\fR
When recursing directories, don\'t scan dirs that reside on other storage devices\. This lets you avoid scanning slow network mounts\. This feature is not supported on all platforms\.
.
.TP
\fB\-p \-\-path\-to\-ignore STRING\fR
Provide a path to a specific \.ignore file\.
.
.TP
\fB\-\-pager COMMAND\fR
Use a pager such as \fBless\fR\. Use \fB\-\-nopager\fR to override\. This option is also ignored if output is piped to another program\.
.
.TP
\fB\-\-parallel\fR
Parse the input stream as a search term, not data to search\. This is meant to be used with tools such as GNU parallel\. For example: \fBecho "foo\enbar\enbaz" | parallel "ag {} \."\fR will run 3 instances of ag, searching the current directory for "foo", "bar", and "baz"\.
.
.TP
\fB\-\-print\-long\-lines\fR
Print matches on very long lines (> 2k characters by default)\.
.
.TP
\fB\-\-passthrough \-\-passthru\fR
When searching a stream, print all lines even if they don\'t match\.
.
.TP
\fB\-Q \-\-literal\fR
Do not parse PATTERN as a regular expression\. Try to match it literally\.
.
.TP
\fB\-r \-\-recurse\fR
Recurse into directories when searching\. Default is true\.
.
.TP
\fB\-s \-\-case\-sensitive\fR
Match case\-sensitively\.
.
.TP
\fB\-S \-\-smart\-case\fR
Match case\-sensitively if there are any uppercase letters in PATTERN, case\-insensitively otherwise\. Enabled by default\.
.
.TP
\fB\-\-search\-binary\fR
Search binary files for matches\.
.
.TP
\fB\-\-silent\fR
Suppress all log messages, including errors\.
.
.TP
\fB\-\-stats\fR
Print stats (files scanned, time taken, etc)\.
.
.TP
\fB\-\-stats\-only\fR
Print stats (files scanned, time taken, etc) and nothing else\.
.
.TP
\fB\-t \-\-all\-text\fR
Search all text files\. This doesn\'t include hidden files\.
.
.TP
\fB\-u \-\-unrestricted\fR
Search \fIall\fR files\. This ignores \.ignore, \.gitignore, etc\. It searches binary and hidden files as well\.
.
.TP
\fB\-U \-\-skip\-vcs\-ignores\fR
Ignore VCS ignore files (\.gitignore, \.hgignore), but still use \.ignore\.
.
.TP
\fB\-v \-\-invert\-match\fR
Match every line \fInot\fR containing the specified pattern\.
.
.TP
\fB\-V \-\-version\fR
Print version info\.
.
.TP
\fB\-\-vimgrep\fR
Output results in the same form as Vim\'s \fB:vimgrep /pattern/g\fR
.
.IP
Here is a ~/\.vimrc configuration example:
.
.IP
\fBset grepprg=ag\e \-\-vimgrep\e $*\fR \fBset grepformat=%f:%l:%c:%m\fR
.
.IP
Then use \fB:grep\fR to grep for something\. Then use \fB:copen\fR, \fB:cn\fR, \fB:cp\fR, etc\. to navigate through the matches\.
.
.TP
\fB\-w \-\-word\-regexp\fR
Only match whole words\.
.
.TP
\fB\-\-workers NUM\fR
Use NUM worker threads\. Default is the number of CPU cores, with a max of 8\.
.
.TP
\fB\-z \-\-search\-zip\fR
Search contents of compressed files\. Currently, gz and xz are supported\. This option requires that ag is built with lzma and zlib\.
.
.TP
\fB\-0 \-\-null \-\-print0\fR
Separate the filenames with \fB\e0\fR, rather than \fB\en\fR: this allows \fBxargs \-0 <command>\fR to correctly process filenames containing spaces or newlines\.
.
.SH "FILE TYPES"
It is possible to restrict the types of files searched\. For example, passing \fB\-\-html\fR will search only files with the extensions \fBhtm\fR, \fBhtml\fR, \fBshtml\fR or \fBxhtml\fR\. For a list of supported types, run \fBag \-\-list\-file\-types\fR\.
.
.SH "IGNORING FILES"
By default, ag will ignore files whose names match patterns in \.gitignore, \.hgignore, or \.ignore\. These files can be anywhere in the directories being searched\. Binary files are ignored by default as well\. Finally, ag looks in $HOME/\.agignore for ignore patterns\.
.
.P
If you want to ignore \.gitignore and \.hgignore, but still take \.ignore into account, use \fB\-U\fR\.
.
.P
Use the \fB\-t\fR option to search all text files; \fB\-a\fR to search all files; and \fB\-u\fR to search all, including hidden files\.
.
.SH "EXAMPLES"
\fBag printf\fR: Find matches for "printf" in the current directory\.
.
.P
\fBag foo /bar/\fR: Find matches for "foo" in path /bar/\.
.
.P
\fBag \-\- \-\-foo\fR: Find matches for "\-\-foo" in the current directory\. (As with most UNIX command line utilities, "\-\-" is used to signify that the remaining arguments should not be treated as options\.)
.
.SH "ABOUT"
ag was originally created by Geoff Greer\. More information (and the latest release) can be found at http://geoff\.greer\.fm/ag
.
.SH "SEE ALSO"
grep(1)
================================================
FILE: doc/ag.1.md
================================================
ag(1) -- The Silver Searcher. Like ack, but faster.
=============================================
## SYNOPSIS
`ag` [_options_] _pattern_ [_path ..._]
## DESCRIPTION
Recursively search for PATTERN in PATH. Like grep or ack, but faster.
## OPTIONS
* `--ackmate`:
Output results in a format parseable by [AckMate](https://github.com/protocool/AckMate).
* `--[no]affinity`:
Set thread affinity (if platform supports it). Default is true.
* `-a --all-types`:
Search all files. This doesn't include hidden files, and doesn't respect any ignore files.
* `-A --after [LINES]`:
Print lines after match. If not provided, LINES defaults to 2.
* `-B --before [LINES]`:
Print lines before match. If not provided, LINES defaults to 2.
* `--[no]break`:
Print a newline between matches in different files. Enabled by default.
* `-c --count`:
Only print the number of matches in each file.
Note: This is the number of matches, **not** the number of matching lines.
Pipe output to `wc -l` if you want the number of matching lines.
* `--[no]color`:
Print color codes in results. Enabled by default.
* `--color-line-number`:
Color codes for line numbers. Default is 1;33.
* `--color-match`:
Color codes for result match numbers. Default is 30;43.
* `--color-path`:
Color codes for path names. Default is 1;32.
* `--column`:
Print column numbers in results.
* `-C --context [LINES]`:
Print lines before and after matches. Default is 2.
* `-D --debug`:
Output ridiculous amounts of debugging info. Not useful unless you're actually debugging.
* `--depth NUM`:
Search up to NUM directories deep, -1 for unlimited. Default is 25.
* `--[no]filename`:
Print file names. Enabled by default, except when searching a single file.
* `-f --[no]follow`:
Follow symlinks. Default is false.
* `-F --fixed-strings`:
Alias for --literal for compatibility with grep.
* `--[no]group`:
The default, `--group`, lumps multiple matches in the same file
together, and presents them under a single occurrence of the
filename. `--nogroup` refrains from this, and instead places the
filename at the start of each match line.
* `-g PATTERN`:
Print filenames matching PATTERN.
* `-G --file-search-regex PATTERN`:
Only search files whose names match PATTERN.
* `-H --[no]heading`:
Print filenames above matching contents.
* `--hidden`:
Search hidden files. This option obeys ignored files.
* `--ignore PATTERN`:
Ignore files/directories whose names match this pattern. Literal
file and directory names are also allowed.
* `--ignore-dir NAME`:
Alias for --ignore for compatibility with ack.
* `-i --ignore-case`:
Match case-insensitively.
* `-l --files-with-matches`:
Only print the names of files containing matches, not the matching
lines. An empty query will print all files that would be searched.
* `-L --files-without-matches`:
Only print the names of files that don't contain matches.
* `--list-file-types`:
See `FILE TYPES` below.
* `-m --max-count NUM`:
Skip the rest of a file after NUM matches. Default is 0, which never skips.
* `--[no]mmap`:
Toggle use of memory-mapped I/O. Defaults to true on platforms where
`mmap()` is faster than `read()`. (All but macOS.)
* `--[no]multiline`:
Match regexes across newlines. Enabled by default.
* `-n --norecurse`:
Don't recurse into directories.
* `--[no]numbers`:
Print line numbers. Default is to omit line numbers when searching streams.
* `-o --only-matching`:
Print only the matching part of the lines.
* `--one-device`:
When recursing directories, don't scan dirs that reside on other storage
devices. This lets you avoid scanning slow network mounts.
This feature is not supported on all platforms.
* `-p --path-to-ignore STRING`:
Provide a path to a specific .ignore file.
* `--pager COMMAND`:
Use a pager such as `less`. Use `--nopager` to override. This option
is also ignored if output is piped to another program.
* `--parallel`:
Parse the input stream as a search term, not data to search. This is meant
to be used with tools such as GNU parallel. For example:
`echo "foo\nbar\nbaz" | parallel "ag {} ."` will run 3 instances of ag,
searching the current directory for "foo", "bar", and "baz".
* `--print-long-lines`:
Print matches on very long lines (> 2k characters by default).
* `--passthrough --passthru`:
When searching a stream, print all lines even if they don't match.
* `-Q --literal`:
Do not parse PATTERN as a regular expression. Try to match it literally.
* `-r --recurse`:
Recurse into directories when searching. Default is true.
* `-s --case-sensitive`:
Match case-sensitively.
* `-S --smart-case`:
Match case-sensitively if there are any uppercase letters in PATTERN,
case-insensitively otherwise. Enabled by default.
* `--search-binary`:
Search binary files for matches.
* `--silent`:
Suppress all log messages, including errors.
* `--stats`:
Print stats (files scanned, time taken, etc).
* `--stats-only`:
Print stats (files scanned, time taken, etc) and nothing else.
* `-t --all-text`:
Search all text files. This doesn't include hidden files.
* `-u --unrestricted`:
Search *all* files. This ignores .ignore, .gitignore, etc. It searches
binary and hidden files as well.
* `-U --skip-vcs-ignores`:
Ignore VCS ignore files (.gitignore, .hgignore), but still
use .ignore.
* `-v --invert-match`:
Match every line *not* containing the specified pattern.
* `-V --version`:
Print version info.
* `--vimgrep`:
Output results in the same form as Vim's `:vimgrep /pattern/g`
Here is a ~/.vimrc configuration example:
`set grepprg=ag\ --vimgrep\ $*`
`set grepformat=%f:%l:%c:%m`
Then use `:grep` to grep for something.
Then use `:copen`, `:cn`, `:cp`, etc. to navigate through the matches.
* `-w --word-regexp`:
Only match whole words.
* `--workers NUM`:
Use NUM worker threads. Default is the number of CPU cores, with a max of 8.
* `-W --width NUM`:
Truncate match lines after NUM characters.
* `-z --search-zip`:
Search contents of compressed files. Currently, gz and xz are supported.
This option requires that ag is built with lzma and zlib.
* `-0 --null --print0`:
Separate the filenames with `\0`, rather than `\n`:
this allows `xargs -0 <command>` to correctly process filenames containing
spaces or newlines.
## FILE TYPES
It is possible to restrict the types of files searched. For example, passing
`--html` will search only files with the extensions `htm`, `html`, `shtml`
or `xhtml`. For a list of supported types, run `ag --list-file-types`.
## IGNORING FILES
By default, ag will ignore files whose names match patterns in .gitignore,
.hgignore, or .ignore. These files can be anywhere in the directories being
searched. Binary files are ignored by default as well. Finally, ag looks in
$HOME/.agignore for ignore patterns.
If you want to ignore .gitignore and .hgignore, but still take .ignore into
account, use `-U`.
Use the `-t` option to search all text files; `-a` to search all files; and `-u`
to search all, including hidden files.
## EXAMPLES
`ag printf`:
Find matches for "printf" in the current directory.
`ag foo /bar/`:
Find matches for "foo" in path /bar/.
`ag -- --foo`:
Find matches for "--foo" in the current directory. (As with most UNIX command
line utilities, "--" is used to signify that the remaining arguments should
not be treated as options.)
## ABOUT
ag was originally created by Geoff Greer. More information (and the latest
release) can be found at http://geoff.greer.fm/ag
## SEE ALSO
grep(1)
================================================
FILE: doc/generate_man.sh
================================================
#!/bin/sh
# ronn is used to turn the markdown into a manpage.
# Get ronn at https://github.com/rtomayko/ronn
# Alternately, since ronn is a Ruby gem, you can just
# `gem install ronn`
sed -e 's/\\0/\\\\0/' <ag.1.md >ag.1.md.tmp
ronn -r ag.1.md.tmp
rm -f ag.1.md.tmp
================================================
FILE: format.sh
================================================
#!/bin/bash
function usage() {
echo "Usage: $0 test|reformat"
}
if [ $# -eq 0 ]
then
usage
exit 0
fi
if [ -z "$CLANG_FORMAT" ]
then
CLANG_FORMAT=clang-format
echo "No CLANG_FORMAT set. Using $CLANG_FORMAT"
fi
if ! type "$CLANG_FORMAT" &> /dev/null
then
echo "The command \"$CLANG_FORMAT\" was not found"
exit 1
fi
SOURCE_FILES=$(git ls-files src/)
if [ "$1" == "reformat" ]
then
echo "Reformatting source files"
# shellcheck disable=2086
echo $CLANG_FORMAT -style=file -i $SOURCE_FILES
# shellcheck disable=2086
$CLANG_FORMAT -style=file -i $SOURCE_FILES
exit 0
elif [ "$1" == "test" ]
then
# shellcheck disable=2086
RESULT=$($CLANG_FORMAT -style=file -output-replacements-xml $SOURCE_FILES | grep -c '<replacement ')
if [ "$RESULT" -eq 0 ]
then
echo "code is formatted correctly :)"
exit 0
else
echo "code is not formatted correctly! :("
echo "Suggested change:"
cp -r src clang_format_src
$CLANG_FORMAT -style=file -i clang_format_src/*.c clang_format_src/*.h
diff -ur src clang_format_src
rm -r clang_format_src
echo "Run '$0 reformat' to fix formatting"
exit 1
fi
else
echo "invalid command: $1"
usage
exit 1
fi
================================================
FILE: m4/ax_pthread.m4
================================================
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_pthread.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_PTHREAD([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]])
#
# DESCRIPTION
#
# This macro figures out how to build C programs using POSIX threads. It
# sets the PTHREAD_LIBS output variable to the threads library and linker
# flags, and the PTHREAD_CFLAGS output variable to any special C compiler
# flags that are needed. (The user can also force certain compiler
# flags/libs to be tested by setting these environment variables.)
#
# Also sets PTHREAD_CC to any special C compiler that is needed for
# multi-threaded programs (defaults to the value of CC otherwise). (This
# is necessary on AIX to use the special cc_r compiler alias.)
#
# NOTE: You are assumed to not only compile your program with these flags,
# but also link it with them as well. e.g. you should link with
# $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
#
# If you are only building threads programs, you may wish to use these
# variables in your default LIBS, CFLAGS, and CC:
#
# LIBS="$PTHREAD_LIBS $LIBS"
# CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
# CC="$PTHREAD_CC"
#
# In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute constant
# has a nonstandard name, defines PTHREAD_CREATE_JOINABLE to that name
# (e.g. PTHREAD_CREATE_UNDETACHED on AIX).
#
# Also HAVE_PTHREAD_PRIO_INHERIT is defined if pthread is found and the
# PTHREAD_PRIO_INHERIT symbol is defined when compiling with
# PTHREAD_CFLAGS.
#
# ACTION-IF-FOUND is a list of shell commands to run if a threads library
# is found, and ACTION-IF-NOT-FOUND is a list of commands to run it if it
# is not found. If ACTION-IF-FOUND is not specified, the default action
# will define HAVE_PTHREAD.
#
# Please let the authors know if this macro fails on any platform, or if
# you have any other suggestions or comments. This macro was based on work
# by SGJ on autoconf scripts for FFTW (http://www.fftw.org/) (with help
# from M. Frigo), as well as ac_pthread and hb_pthread macros posted by
# Alejandro Forero Cuervo to the autoconf macro repository. We are also
# grateful for the helpful feedback of numerous users.
#
# Updated for Autoconf 2.68 by Daniel Richard G.
#
# LICENSE
#
# Copyright (c) 2008 Steven G. Johnson <stevenj@alum.mit.edu>
# Copyright (c) 2011 Daniel Richard G. <skunk@iSKUNK.ORG>
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
# Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
#
# As a special exception, the respective Autoconf Macro's copyright owner
# gives unlimited permission to copy, distribute and modify the configure
# scripts that are the output of Autoconf when processing the Macro. You
# need not follow the terms of the GNU General Public License when using
# or distributing such scripts, even though portions of the text of the
# Macro appear in them. The GNU General Public License (GPL) does govern
# all other use of the material that constitutes the Autoconf Macro.
#
# This special exception to the GPL applies to versions of the Autoconf
# Macro released by the Autoconf Archive. When you make and distribute a
# modified version of the Autoconf Macro, you may extend this special
# exception to the GPL to apply to your modified version as well.
#serial 21
AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
AC_DEFUN([AX_PTHREAD], [
AC_REQUIRE([AC_CANONICAL_HOST])
AC_LANG_PUSH([C])
ax_pthread_ok=no
# We used to check for pthread.h first, but this fails if pthread.h
# requires special compiler flags (e.g. on True64 or Sequent).
# It gets checked for in the link test anyway.
# First of all, check if the user has set any of the PTHREAD_LIBS,
# etcetera environment variables, and if threads linking works using
# them:
if test x"$PTHREAD_LIBS$PTHREAD_CFLAGS" != x; then
save_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
save_LIBS="$LIBS"
LIBS="$PTHREAD_LIBS $LIBS"
AC_MSG_CHECKING([for pthread_join in LIBS=$PTHREAD_LIBS with CFLAGS=$PTHREAD_CFLAGS])
AC_TRY_LINK_FUNC([pthread_join], [ax_pthread_ok=yes])
AC_MSG_RESULT([$ax_pthread_ok])
if test x"$ax_pthread_ok" = xno; then
PTHREAD_LIBS=""
PTHREAD_CFLAGS=""
fi
LIBS="$save_LIBS"
CFLAGS="$save_CFLAGS"
fi
# We must check for the threads library under a number of different
# names; the ordering is very important because some systems
# (e.g. DEC) have both -lpthread and -lpthreads, where one of the
# libraries is broken (non-POSIX).
# Create a list of thread flags to try. Items starting with a "-" are
# C compiler flags, and other items are library names, except for "none"
# which indicates that we try without any flags at all, and "pthread-config"
# which is a program returning the flags for the Pth emulation library.
ax_pthread_flags="pthreads none -Kthread -kthread lthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config"
# The ordering *is* (sometimes) important. Some notes on the
# individual items follow:
# pthreads: AIX (must check this before -lpthread)
# none: in case threads are in libc; should be tried before -Kthread and
# other compiler flags to prevent continual compiler warnings
# -Kthread: Sequent (threads in libc, but -Kthread needed for pthread.h)
# -kthread: FreeBSD kernel threads (preferred to -pthread since SMP-able)
# lthread: LinuxThreads port on FreeBSD (also preferred to -pthread)
# -pthread: Linux/gcc (kernel threads), BSD/gcc (userland threads)
# -pthreads: Solaris/gcc
# -mthreads: Mingw32/gcc, Lynx/gcc
# -mt: Sun Workshop C (may only link SunOS threads [-lthread], but it
# doesn't hurt to check since this sometimes defines pthreads too;
# also defines -D_REENTRANT)
# ... -mt is also the pthreads flag for HP/aCC
# pthread: Linux, etcetera
# --thread-safe: KAI C++
# pthread-config: use pthread-config program (for GNU Pth library)
case ${host_os} in
solaris*)
# On Solaris (at least, for some versions), libc contains stubbed
# (non-functional) versions of the pthreads routines, so link-based
# tests will erroneously succeed. (We need to link with -pthreads/-mt/
# -lpthread.) (The stubs are missing pthread_cleanup_push, or rather
# a function called by this macro, so we could check for that, but
# who knows whether they'll stub that too in a future libc.) So,
# we'll just look for -pthreads and -lpthread first:
ax_pthread_flags="-pthreads pthread -mt -pthread $ax_pthread_flags"
;;
darwin*)
ax_pthread_flags="-pthread $ax_pthread_flags"
;;
esac
# Clang doesn't consider unrecognized options an error unless we specify
# -Werror. We throw in some extra Clang-specific options to ensure that
# this doesn't happen for GCC, which also accepts -Werror.
AC_MSG_CHECKING([if compiler needs -Werror to reject unknown flags])
save_CFLAGS="$CFLAGS"
ax_pthread_extra_flags="-Werror"
CFLAGS="$CFLAGS $ax_pthread_extra_flags -Wunknown-warning-option -Wsizeof-array-argument"
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([int foo(void);],[foo()])],
[AC_MSG_RESULT([yes])],
[ax_pthread_extra_flags=
AC_MSG_RESULT([no])])
CFLAGS="$save_CFLAGS"
if test x"$ax_pthread_ok" = xno; then
for flag in $ax_pthread_flags; do
case $flag in
none)
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
-*)
AC_MSG_CHECKING([whether pthreads work with $flag])
PTHREAD_CFLAGS="$flag"
;;
pthread-config)
AC_CHECK_PROG([ax_pthread_config], [pthread-config], [yes], [no])
if test x"$ax_pthread_config" = xno; then continue; fi
PTHREAD_CFLAGS="`pthread-config --cflags`"
PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`"
;;
*)
AC_MSG_CHECKING([for the pthreads library -l$flag])
PTHREAD_LIBS="-l$flag"
;;
esac
save_LIBS="$LIBS"
save_CFLAGS="$CFLAGS"
LIBS="$PTHREAD_LIBS $LIBS"
CFLAGS="$CFLAGS $PTHREAD_CFLAGS $ax_pthread_extra_flags"
# Check for various functions. We must include pthread.h,
# since some functions may be macros. (On the Sequent, we
# need a special flag -Kthread to make this header compile.)
# We check for pthread_join because it is in -lpthread on IRIX
# while pthread_create is in libc. We check for pthread_attr_init
# due to DEC craziness with -lpthreads. We check for
# pthread_cleanup_push because it is one of the few pthread
# functions on Solaris that doesn't have a non-functional libc stub.
# We try pthread_create on general principles.
AC_LINK_IFELSE([AC_LANG_PROGRAM([#include <pthread.h>
static void routine(void *a) { a = 0; }
static void *start_routine(void *a) { return a; }],
[pthread_t th; pthread_attr_t attr;
pthread_create(&th, 0, start_routine, 0);
pthread_join(th, 0);
pthread_attr_init(&attr);
pthread_cleanup_push(routine, 0);
pthread_cleanup_pop(0) /* ; */])],
[ax_pthread_ok=yes],
[])
LIBS="$save_LIBS"
CFLAGS="$save_CFLAGS"
AC_MSG_RESULT([$ax_pthread_ok])
if test "x$ax_pthread_ok" = xyes; then
break;
fi
PTHREAD_LIBS=""
PTHREAD_CFLAGS=""
done
fi
# Various other checks:
if test "x$ax_pthread_ok" = xyes; then
save_LIBS="$LIBS"
LIBS="$PTHREAD_LIBS $LIBS"
save_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
# Detect AIX lossage: JOINABLE attribute is called UNDETACHED.
AC_MSG_CHECKING([for joinable pthread attribute])
attr_name=unknown
for attr in PTHREAD_CREATE_JOINABLE PTHREAD_CREATE_UNDETACHED; do
AC_LINK_IFELSE([AC_LANG_PROGRAM([#include <pthread.h>],
[int attr = $attr; return attr /* ; */])],
[attr_name=$attr; break],
[])
done
AC_MSG_RESULT([$attr_name])
if test "$attr_name" != PTHREAD_CREATE_JOINABLE; then
AC_DEFINE_UNQUOTED([PTHREAD_CREATE_JOINABLE], [$attr_name],
[Define to necessary symbol if this constant
uses a non-standard name on your system.])
fi
AC_MSG_CHECKING([if more special flags are required for pthreads])
flag=no
case ${host_os} in
aix* | freebsd* | darwin*) flag="-D_THREAD_SAFE";;
osf* | hpux*) flag="-D_REENTRANT";;
solaris*)
if test "$GCC" = "yes"; then
flag="-D_REENTRANT"
else
# TODO: What about Clang on Solaris?
flag="-mt -D_REENTRANT"
fi
;;
esac
AC_MSG_RESULT([$flag])
if test "x$flag" != xno; then
PTHREAD_CFLAGS="$flag $PTHREAD_CFLAGS"
fi
AC_CACHE_CHECK([for PTHREAD_PRIO_INHERIT],
[ax_cv_PTHREAD_PRIO_INHERIT], [
AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <pthread.h>]],
[[int i = PTHREAD_PRIO_INHERIT;]])],
[ax_cv_PTHREAD_PRIO_INHERIT=yes],
[ax_cv_PTHREAD_PRIO_INHERIT=no])
])
AS_IF([test "x$ax_cv_PTHREAD_PRIO_INHERIT" = "xyes"],
[AC_DEFINE([HAVE_PTHREAD_PRIO_INHERIT], [1], [Have PTHREAD_PRIO_INHERIT.])])
LIBS="$save_LIBS"
CFLAGS="$save_CFLAGS"
# More AIX lossage: compile with *_r variant
if test "x$GCC" != xyes; then
case $host_os in
aix*)
AS_CASE(["x/$CC"],
[x*/c89|x*/c89_128|x*/c99|x*/c99_128|x*/cc|x*/cc128|x*/xlc|x*/xlc_v6|x*/xlc128|x*/xlc128_v6],
[#handle absolute path differently from PATH based program lookup
AS_CASE(["x$CC"],
[x/*],
[AS_IF([AS_EXECUTABLE_P([${CC}_r])],[PTHREAD_CC="${CC}_r"])],
[AC_CHECK_PROGS([PTHREAD_CC],[${CC}_r],[$CC])])])
;;
esac
fi
fi
test -n "$PTHREAD_CC" || PTHREAD_CC="$CC"
AC_SUBST([PTHREAD_LIBS])
AC_SUBST([PTHREAD_CFLAGS])
AC_SUBST([PTHREAD_CC])
# Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND:
if test x"$ax_pthread_ok" = xyes; then
ifelse([$1],,[AC_DEFINE([HAVE_PTHREAD],[1],[Define if you have POSIX threads libraries and header files.])],[$1])
:
else
ax_pthread_ok=no
$2
fi
AC_LANG_POP
])dnl AX_PTHREAD
================================================
FILE: pgo.sh
================================================
#!/bin/sh
set -e
cd "$(dirname "$0")"
make clean
./build.sh CFLAGS="$CFLAGS -fprofile-generate"
./ag example ..
make clean
./build.sh CFLAGS="$CFLAGS -fprofile-correction -fprofile-use"
================================================
FILE: sanitize.sh
================================================
#!/bin/bash
# Copyright 2016 Allen Wild
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
AVAILABLE_SANITIZERS=(
address
thread
undefined
valgrind
)
DEFAULT_SANITIZERS=(
address
thread
undefined
)
usage() {
cat <<EOF
Usage: $0 [-h] [valgrind | [SANITIZERS ...]]
This script recompiles ag using -fsanitize=<SANITIZER> and then runs the test suite.
Memory leaks or other errors will be printed in ag's output, thus failing the test.
Available LLVM sanitizers are: ${AVAILABLE_SANITIZERS[*]}
The compile-time sanitizers are supported in clang/llvm >= 3.1 and gcc >= 4.8
for x86_64 Linux only. clang is preferred and will be used, if available.
For function names and line numbers in error output traces, llvm-symbolizer needs
to be available in PATH or set through ASAN_SYMBOLIZER_PATH.
If 'valgrind' is passed as the sanitizer, then ag will be run through valgrind
without recompiling. If $(dirname $0)/ag doesn't exist, then it will be built.
WARNING: This script will run "make distclean" and "./configure" to recompile ag
once per sanitizer (except for valgrind). If you need to pass additional
options to ./configure, put them in the CONFIGOPTS environment variable.
EOF
}
vrun() {
echo "Running: $*"
"$@"
}
die() {
echo "Fatal: $*"
exit 1
}
valid_sanitizer() {
for san in "${AVAILABLE_SANITIZERS[@]}"; do
if [[ "$1" == "$san" ]]; then
return 0
fi
done
return 1
}
run_sanitizer() {
sanitizer=$1
if [[ "$sanitizer" == "valgrind" ]]; then
run_valgrind
return $?
fi
echo -e "\nCompiling for sanitizer '$sanitizer'"
[[ -f Makefile ]] && vrun make distclean
vrun ./configure $CONFIGOPTS CC=$SANITIZE_CC \
CFLAGS="-g -O0 -fsanitize=$sanitizer $EXTRA_CFLAGS"
if [[ $? != 0 ]]; then
echo "ERROR: Failed to configure. Try setting CONFIGOPTS?"
return 1
fi
vrun make
if [[ $? != 0 ]]; then
echo "ERROR: failed to build"
return 1
fi
echo "Testing with sanitizer '$sanitizer'"
vrun make test
if [[ $? != 0 ]]; then
echo "Tests for sanitizer '$sanitizer' FAIL!"
echo "Check the above output for failure information"
return 2
else
echo "Tests for sanitizer '$sanitizer' PASS!"
return 0
fi
}
run_valgrind() {
echo "Compiling ag normally for use with valgrind"
[[ -f Makefile ]] && vrun make distclean
vrun ./configure $CONFIGOPTS
if [[ $? != 0 ]]; then
echo "ERROR: Failed to configure. Try setting CONFIGOPTS?"
return 1
fi
vrun make
if [[ $? != 0 ]]; then
echo "ERROR: failed to build"
return 1
fi
echo "Running: AGPROG=\"valgrind -q $PWD/ag\" make test"
AGPROG="valgrind -q $PWD/ag" make test
if [[ $? != 0 ]]; then
echo "Valgrind tests FAIL!"
return 1
else
echo "Valgrind tests PASS!"
return 0
fi
}
#### MAIN ####
run_sanitizers=()
for opt in "$@"; do
if [[ "$opt" == -* ]]; then
case opt in
-h|--help)
usage
exit 0
;;
*)
echo "Unknown option: '$opt'"
usage
exit 1
;;
esac
else
if valid_sanitizer "$opt"; then
run_sanitizers+=("$opt")
else
echo "Invalid Sanitizer: '$opt'"
usage
exit 1
fi
fi
done
if [[ ${#run_sanitizers[@]} == 0 ]]; then
run_sanitizers=(${DEFAULT_SANITIZERS[@]})
fi
if [[ -n $CC ]]; then
echo "Using CC=$CC"
SANITIZE_CC="$CC"
elif which clang &>/dev/null; then
SANITIZE_CC="clang"
else
echo "Warning: CC unset and clang not found"
fi
if [[ -n $CFLAGS ]]; then
EXTRA_CFLAGS="$CFLAGS"
unset CFLAGS
fi
if [[ ! -e ./configure ]]; then
echo "Warning: ./configure not found. Running autogen"
vrun ./autogen.sh || die "autogen.sh failed"
fi
echo "Running sanitizers: ${run_sanitizers[*]}"
failedsan=()
for san in "${run_sanitizers[@]}"; do
run_sanitizer $san
if [[ $? != 0 ]]; then
failedsan+=($san)
fi
done
if [[ ${#failedsan[@]} == 0 ]]; then
echo "All sanitizers PASSED"
exit 0
else
echo "The following sanitizers FAILED: ${failedsan[*]}"
exit ${#failedsan[@]}
fi
================================================
FILE: src/decompress.c
================================================
#include <string.h>
#include <unistd.h>
#include "decompress.h"
#ifdef HAVE_LZMA_H
#include <lzma.h>
/* http://tukaani.org/xz/xz-file-format.txt */
const uint8_t XZ_HEADER_MAGIC[6] = { 0xFD, '7', 'z', 'X', 'Z', 0x00 };
const uint8_t LZMA_HEADER_SOMETIMES[3] = { 0x5D, 0x00, 0x00 };
#endif
#ifdef HAVE_ZLIB_H
#define ZLIB_CONST 1
#include <zlib.h>
/* Code in decompress_zlib from
*
* https://raw.github.com/madler/zlib/master/examples/zpipe.c
*
* zpipe.c: example of proper use of zlib's inflate() and deflate()
* Not copyrighted -- provided to the public domain
* Version 1.4 11 December 2005 Mark Adler
*/
static void *decompress_zlib(const void *buf, const int buf_len,
const char *dir_full_path, int *new_buf_len) {
int ret = 0;
unsigned char *result = NULL;
size_t result_size = 0;
size_t pagesize = 0;
z_stream stream;
log_debug("Decompressing zlib file %s", dir_full_path);
/* allocate inflate state */
stream.zalloc = Z_NULL;
stream.zfree = Z_NULL;
stream.opaque = Z_NULL;
stream.avail_in = 0;
stream.next_in = Z_NULL;
/* Add 32 to allow zlib and gzip format detection */
if (inflateInit2(&stream, 32 + 15) != Z_OK) {
log_err("Unable to initialize zlib: %s", stream.msg);
goto error_out;
}
stream.avail_in = buf_len;
/* Explicitly cast away the const-ness of buf */
stream.next_in = (Bytef *)buf;
pagesize = getpagesize();
result_size = ((buf_len + pagesize - 1) & ~(pagesize - 1));
do {
do {
unsigned char *tmp_result = result;
/* Double the buffer size and realloc */
result_size *= 2;
result = (unsigned char *)realloc(result, result_size * sizeof(unsigned char));
if (result == NULL) {
free(tmp_result);
log_err("Unable to allocate %d bytes to decompress file %s", result_size * sizeof(unsigned char), dir_full_path);
inflateEnd(&stream);
goto error_out;
}
stream.avail_out = result_size / 2;
stream.next_out = &result[stream.total_out];
ret = inflate(&stream, Z_SYNC_FLUSH);
log_debug("inflate ret = %d", ret);
switch (ret) {
case Z_STREAM_ERROR: {
log_err("Found stream error while decompressing zlib stream: %s", stream.msg);
inflateEnd(&stream);
goto error_out;
}
case Z_NEED_DICT:
case Z_DATA_ERROR:
case Z_MEM_ERROR: {
log_err("Found mem/data error while decompressing zlib stream: %s", stream.msg);
inflateEnd(&stream);
goto error_out;
}
}
} while (stream.avail_out == 0);
} while (ret == Z_OK);
*new_buf_len = stream.total_out;
inflateEnd(&stream);
if (ret == Z_STREAM_END) {
return result;
}
error_out:
*new_buf_len = 0;
return NULL;
}
#endif
static void *decompress_lzw(const void *buf, const int buf_len,
const char *dir_full_path, int *new_buf_len) {
(void)buf;
(void)buf_len;
log_err("LZW (UNIX compress) files not yet supported: %s", dir_full_path);
*new_buf_len = 0;
return NULL;
}
static void *decompress_zip(const void *buf, const int buf_len,
const char *dir_full_path, int *new_buf_len) {
(void)buf;
(void)buf_len;
log_err("Zip files not yet supported: %s", dir_full_path);
*new_buf_len = 0;
return NULL;
}
#ifdef HAVE_LZMA_H
static void *decompress_lzma(const void *buf, const int buf_len,
const char *dir_full_path, int *new_buf_len) {
lzma_stream stream = LZMA_STREAM_INIT;
lzma_ret lzrt;
unsigned char *result = NULL;
size_t result_size = 0;
size_t pagesize = 0;
stream.avail_in = buf_len;
stream.next_in = buf;
lzrt = lzma_auto_decoder(&stream, -1, 0);
if (lzrt != LZMA_OK) {
log_err("Unable to initialize lzma_auto_decoder: %d", lzrt);
goto error_out;
}
pagesize = getpagesize();
result_size = ((buf_len + pagesize - 1) & ~(pagesize - 1));
do {
do {
unsigned char *tmp_result = result;
/* Double the buffer size and realloc */
result_size *= 2;
result = (unsigned char *)realloc(result, result_size * sizeof(unsigned char));
if (result == NULL) {
free(tmp_result);
log_err("Unable to allocate %d bytes to decompress file %s", result_size * sizeof(unsigned char), dir_full_path);
goto error_out;
}
stream.avail_out = result_size / 2;
stream.next_out = &result[stream.total_out];
lzrt = lzma_code(&stream, LZMA_RUN);
log_debug("lzma_code ret = %d", lzrt);
switch (lzrt) {
case LZMA_OK:
case LZMA_STREAM_END:
break;
default:
log_err("Found mem/data error while decompressing xz/lzma stream: %d", lzrt);
goto error_out;
}
} while (stream.avail_out == 0);
} while (lzrt == LZMA_OK);
*new_buf_len = stream.total_out;
if (lzrt == LZMA_STREAM_END) {
lzma_end(&stream);
return result;
}
error_out:
lzma_end(&stream);
*new_buf_len = 0;
if (result) {
free(result);
}
return NULL;
}
#endif
/* This function is very hot. It's called on every file when zip is enabled. */
void *decompress(const ag_compression_type zip_type, const void *buf, const int buf_len,
const char *dir_full_path, int *new_buf_len) {
switch (zip_type) {
#ifdef HAVE_ZLIB_H
case AG_GZIP:
return decompress_zlib(buf, buf_len, dir_full_path, new_buf_len);
#endif
case AG_COMPRESS:
return decompress_lzw(buf, buf_len, dir_full_path, new_buf_len);
case AG_ZIP:
return decompress_zip(buf, buf_len, dir_full_path, new_buf_len);
#ifdef HAVE_LZMA_H
case AG_XZ:
return decompress_lzma(buf, buf_len, dir_full_path, new_buf_len);
#endif
case AG_NO_COMPRESSION:
log_err("File %s is not compressed", dir_full_path);
break;
default:
log_err("Unsupported compression type: %d", zip_type);
}
*new_buf_len = 0;
return NULL;
}
/* This function is very hot. It's called on every file. */
ag_compression_type is_zipped(const void *buf, const int buf_len) {
/* Zip magic numbers
* compressed file: { 0x1F, 0x9B }
* http://en.wikipedia.org/wiki/Compress
*
* gzip file: { 0x1F, 0x8B }
* http://www.gzip.org/zlib/rfc-gzip.html#file-format
*
* zip file: { 0x50, 0x4B, 0x03, 0x04 }
* http://www.pkware.com/documents/casestudies/APPNOTE.TXT (Section 4.3)
*/
const unsigned char *buf_c = buf;
if (buf_len == 0)
return AG_NO_COMPRESSION;
/* Check for gzip & compress */
if (buf_len >= 2) {
if (buf_c[0] == 0x1F) {
if (buf_c[1] == 0x8B) {
#ifdef HAVE_ZLIB_H
log_debug("Found gzip-based stream");
return AG_GZIP;
#endif
} else if (buf_c[1] == 0x9B) {
log_debug("Found compress-based stream");
return AG_COMPRESS;
}
}
}
/* Check for zip */
if (buf_len >= 4) {
if (buf_c[0] == 0x50 && buf_c[1] == 0x4B && buf_c[2] == 0x03 && buf_c[3] == 0x04) {
log_debug("Found zip-based stream");
return AG_ZIP;
}
}
#ifdef HAVE_LZMA_H
if (buf_len >= 6) {
if (memcmp(XZ_HEADER_MAGIC, buf_c, 6) == 0) {
log_debug("Found xz based stream");
return AG_XZ;
}
}
/* LZMA doesn't really have a header: http://www.mail-archive.com/xz-devel@tukaani.org/msg00003.html */
if (buf_len >= 3) {
if (memcmp(LZMA_HEADER_SOMETIMES, buf_c, 3) == 0) {
log_debug("Found lzma-based stream");
return AG_XZ;
}
}
#endif
return AG_NO_COMPRESSION;
}
================================================
FILE: src/decompress.h
================================================
#ifndef DECOMPRESS_H
#define DECOMPRESS_H
#include <stdio.h>
#include "config.h"
#include "log.h"
#include "options.h"
typedef enum {
AG_NO_COMPRESSION,
AG_GZIP,
AG_COMPRESS,
AG_ZIP,
AG_XZ,
} ag_compression_type;
ag_compression_type is_zipped(const void *buf, const int buf_len);
void *decompress(const ag_compression_type zip_type, const void *buf, const int buf_len, const char *dir_full_path, int *new_buf_len);
#if HAVE_FOPENCOOKIE
FILE *decompress_open(int fd, const char *mode, ag_compression_type ctype);
#endif
#endif
================================================
FILE: src/ignore.c
================================================
#include <ctype.h>
#include <dirent.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include "ignore.h"
#include "log.h"
#include "options.h"
#include "scandir.h"
#include "util.h"
#ifdef _WIN32
#include <shlwapi.h>
#define fnmatch(x, y, z) (!PathMatchSpec(y, x))
#else
#include <fnmatch.h>
const int fnmatch_flags = FNM_PATHNAME;
#endif
ignores *root_ignores;
/* TODO: build a huge-ass list of files we want to ignore by default (build cache stuff, pyc files, etc) */
const char *evil_hardcoded_ignore_files[] = {
".",
"..",
NULL
};
/* Warning: changing the first two strings will break skip_vcs_ignores. */
const char *ignore_pattern_files[] = {
".ignore",
".gitignore",
".git/info/exclude",
".hgignore",
NULL
};
int is_empty(ignores *ig) {
return (ig->extensions_len + ig->names_len + ig->slash_names_len + ig->regexes_len + ig->slash_regexes_len == 0);
};
ignores *init_ignore(ignores *parent, const char *dirname, const size_t dirname_len) {
ignores *ig = ag_malloc(sizeof(ignores));
ig->extensions = NULL;
ig->extensions_len = 0;
ig->names = NULL;
ig->names_len = 0;
ig->slash_names = NULL;
ig->slash_names_len = 0;
ig->regexes = NULL;
ig->regexes_len = 0;
ig->invert_regexes = NULL;
ig->invert_regexes_len = 0;
ig->slash_regexes = NULL;
ig->slash_regexes_len = 0;
ig->dirname = dirname;
ig->dirname_len = dirname_len;
if (parent && is_empty(parent) && parent->parent) {
ig->parent = parent->parent;
} else {
ig->parent = parent;
}
if (parent && parent->abs_path_len > 0) {
ag_asprintf(&(ig->abs_path), "%s/%s", parent->abs_path, dirname);
ig->abs_path_len = parent->abs_path_len + 1 + dirname_len;
} else if (dirname_len == 1 && dirname[0] == '.') {
ig->abs_path = ag_malloc(sizeof(char));
ig->abs_path[0] = '\0';
ig->abs_path_len = 0;
} else {
ag_asprintf(&(ig->abs_path), "%s", dirname);
ig->abs_path_len = dirname_len;
}
return ig;
}
void cleanup_ignore(ignores *ig) {
if (ig == NULL) {
return;
}
free_strings(ig->extensions, ig->extensions_len);
free_strings(ig->names, ig->names_len);
free_strings(ig->slash_names, ig->slash_names_len);
free_strings(ig->regexes, ig->regexes_len);
free_strings(ig->invert_regexes, ig->invert_regexes_len);
free_strings(ig->slash_regexes, ig->slash_regexes_len);
if (ig->abs_path) {
free(ig->abs_path);
}
free(ig);
}
void add_ignore_pattern(ignores *ig, const char *pattern) {
int i;
int pattern_len;
/* Strip off the leading dot so that matches are more likely. */
if (strncmp(pattern, "./", 2) == 0) {
pattern++;
}
/* Kill trailing whitespace */
for (pattern_len = strlen(pattern); pattern_len > 0; pattern_len--) {
if (!isspace(pattern[pattern_len - 1])) {
break;
}
}
if (pattern_len == 0) {
log_debug("Pattern is empty. Not adding any ignores.");
return;
}
char ***patterns_p;
size_t *patterns_len;
if (is_fnmatch(pattern)) {
if (pattern[0] == '*' && pattern[1] == '.' && strchr(pattern + 2, '.') && !is_fnmatch(pattern + 2)) {
patterns_p = &(ig->extensions);
patterns_len = &(ig->extensions_len);
pattern += 2;
pattern_len -= 2;
} else if (pattern[0] == '/') {
patterns_p = &(ig->slash_regexes);
patterns_len = &(ig->slash_regexes_len);
pattern++;
pattern_len--;
} else if (pattern[0] == '!') {
patterns_p = &(ig->invert_regexes);
patterns_len = &(ig->invert_regexes_len);
pattern++;
pattern_len--;
} else {
patterns_p = &(ig->regexes);
patterns_len = &(ig->regexes_len);
}
} else {
if (pattern[0] == '/') {
patterns_p = &(ig->slash_names);
patterns_len = &(ig->slash_names_len);
pattern++;
pattern_len--;
} else {
patterns_p = &(ig->names);
patterns_len = &(ig->names_len);
}
}
++*patterns_len;
char **patterns;
/* a balanced binary tree is best for performance, but I'm lazy */
*patterns_p = patterns = ag_realloc(*patterns_p, (*patterns_len) * sizeof(char *));
/* TODO: de-dupe these patterns */
for (i = *patterns_len - 1; i > 0; i--) {
if (strcmp(pattern, patterns[i - 1]) > 0) {
break;
}
patterns[i] = patterns[i - 1];
}
patterns[i] = ag_strndup(pattern, pattern_len);
log_debug("added ignore pattern %s to %s", pattern,
ig == root_ignores ? "root ignores" : ig->abs_path);
}
/* For loading git/hg ignore patterns */
void load_ignore_patterns(ignores *ig, const char *path) {
FILE *fp = NULL;
fp = fopen(path, "r");
if (fp == NULL) {
log_debug("Skipping ignore file %s: not readable", path);
return;
}
log_debug("Loading ignore file %s.", path);
char *line = NULL;
ssize_t line_len = 0;
size_t line_cap = 0;
while ((line_len = getline(&line, &line_cap, fp)) > 0) {
if (line_len == 0 || line[0] == '\n' || line[0] == '#') {
continue;
}
if (line[line_len - 1] == '\n') {
line[line_len - 1] = '\0'; /* kill the \n */
}
add_ignore_pattern(ig, line);
}
free(line);
fclose(fp);
}
static int ackmate_dir_match(const char *dir_name) {
if (opts.ackmate_dir_filter == NULL) {
return 0;
}
/* we just care about the match, not where the matches are */
return pcre_exec(opts.ackmate_dir_filter, NULL, dir_name, strlen(dir_name), 0, 0, NULL, 0);
}
/* This is the hottest code in Ag. 10-15% of all execution time is spent here */
static int path_ignore_search(const ignores *ig, const char *path, const char *filename) {
char *temp;
int temp_start_pos;
size_t i;
int match_pos;
match_pos = binary_search(filename, ig->names, 0, ig->names_len);
if (match_pos >= 0) {
log_debug("file %s ignored because name matches static pattern %s", filename, ig->names[match_pos]);
return 1;
}
ag_asprintf(&temp, "%s/%s", path[0] == '.' ? path + 1 : path, filename);
//ig->abs_path has its leading slash stripped, so we have to strip the leading slash
//of temp as well
temp_start_pos = (temp[0] == '/') ? 1 : 0;
if (strncmp(temp + temp_start_pos, ig->abs_path, ig->abs_path_len) == 0) {
char *slash_filename = temp + temp_start_pos + ig->abs_path_len;
if (slash_filename[0] == '/') {
slash_filename++;
}
match_pos = binary_search(slash_filename, ig->names, 0, ig->names_len);
if (match_pos >= 0) {
log_debug("file %s ignored because name matches static pattern %s", temp, ig->names[match_pos]);
free(temp);
return 1;
}
match_pos = binary_search(slash_filename, ig->slash_names, 0, ig->slash_names_len);
if (match_pos >= 0) {
log_debug("file %s ignored because name matches slash static pattern %s", slash_filename, ig->slash_names[match_pos]);
free(temp);
return 1;
}
for (i = 0; i < ig->names_len; i++) {
char *pos = strstr(slash_filename, ig->names[i]);
if (pos == slash_filename || (pos && *(pos - 1) == '/')) {
pos += strlen(ig->names[i]);
if (*pos == '\0' || *pos == '/') {
log_debug("file %s ignored because path somewhere matches name %s", slash_filename, ig->names[i]);
free(temp);
return 1;
}
}
log_debug("pattern %s doesn't match path %s", ig->names[i], slash_filename);
}
for (i = 0; i < ig->slash_regexes_len; i++) {
if (fnmatch(ig->slash_regexes[i], slash_filename, fnmatch_flags) == 0) {
log_debug("file %s ignored because name matches slash regex pattern %s", slash_filename, ig->slash_regexes[i]);
free(temp);
return 1;
}
log_debug("pattern %s doesn't match slash file %s", ig->slash_regexes[i], slash_filename);
}
}
for (i = 0; i < ig->invert_regexes_len; i++) {
if (fnmatch(ig->invert_regexes[i], filename, fnmatch_flags) == 0) {
log_debug("file %s not ignored because name matches regex pattern !%s", filename, ig->invert_regexes[i]);
free(temp);
return 0;
}
log_debug("pattern !%s doesn't match file %s", ig->invert_regexes[i], filename);
}
for (i = 0; i < ig->regexes_len; i++) {
if (fnmatch(ig->regexes[i], filename, fnmatch_flags) == 0) {
log_debug("file %s ignored because name matches regex pattern %s", filename, ig->regexes[i]);
free(temp);
return 1;
}
log_debug("pattern %s doesn't match file %s", ig->regexes[i], filename);
}
int rv = ackmate_dir_match(temp);
free(temp);
return rv;
}
/* This function is REALLY HOT. It gets called for every file */
int filename_filter(const char *path, const struct dirent *dir, void *baton) {
const char *filename = dir->d_name;
if (!opts.search_hidden_files && filename[0] == '.') {
return 0;
}
size_t i;
for (i = 0; evil_hardcoded_ignore_files[i] != NULL; i++) {
if (strcmp(filename, evil_hardcoded_ignore_files[i]) == 0) {
return 0;
}
}
if (!opts.follow_symlinks && is_symlink(path, dir)) {
log_debug("File %s ignored becaused it's a symlink", dir->d_name);
return 0;
}
if (is_named_pipe(path, dir)) {
log_debug("%s ignored because it's a named pipe or socket", path);
return 0;
}
if (opts.search_all_files && !opts.path_to_ignore) {
return 1;
}
scandir_baton_t *scandir_baton = (scandir_baton_t *)baton;
const char *path_start = scandir_baton->path_start;
const char *extension = strchr(filename, '.');
if (extension) {
if (extension[1]) {
// The dot is not the last character, extension starts at the next one
++extension;
} else {
// No extension
extension = NULL;
}
}
#ifdef HAVE_DIRENT_DNAMLEN
size_t filename_len = dir->d_namlen;
#else
size_t filename_len = 0;
#endif
if (strncmp(filename, "./", 2) == 0) {
#ifndef HAVE_DIRENT_DNAMLEN
filename_len = strlen(filename);
#endif
filename++;
filename_len--;
}
const ignores *ig = scandir_baton->ig;
while (ig != NULL) {
if (extension) {
int match_pos = binary_search(extension, ig->extensions, 0, ig->extensions_len);
if (match_pos >= 0) {
log_debug("file %s ignored because name matches extension %s", filename, ig->extensions[match_pos]);
return 0;
}
}
if (path_ignore_search(ig, path_start, filename)) {
return 0;
}
if (is_directory(path, dir)) {
#ifndef HAVE_DIRENT_DNAMLEN
if (!filename_len) {
filename_len = strlen(filename);
}
#endif
if (filename[filename_len - 1] != '/') {
char *temp;
ag_asprintf(&temp, "%s/", filename);
int rv = path_ignore_search(ig, path_start, temp);
free(temp);
if (rv) {
return 0;
}
}
}
ig = ig->parent;
}
log_debug("%s not ignored", filename);
return 1;
}
================================================
FILE: src/ignore.h
================================================
#ifndef IGNORE_H
#define IGNORE_H
#include <dirent.h>
#include <sys/types.h>
struct ignores {
char **extensions; /* File extensions to ignore */
size_t extensions_len;
char **names; /* Non-regex ignore lines. Sorted so we can binary search them. */
size_t names_len;
char **slash_names; /* Same but starts with a slash */
size_t slash_names_len;
char **regexes; /* For patterns that need fnmatch */
size_t regexes_len;
char **invert_regexes; /* For "!" patterns */
size_t invert_regexes_len;
char **slash_regexes;
size_t slash_regexes_len;
const char *dirname;
size_t dirname_len;
char *abs_path;
size_t abs_path_len;
struct ignores *parent;
};
typedef struct ignores ignores;
extern ignores *root_ignores;
extern const char *evil_hardcoded_ignore_files[];
extern const char *ignore_pattern_files[];
ignores *init_ignore(ignores *parent, const char *dirname, const size_t dirname_len);
void cleanup_ignore(ignores *ig);
void add_ignore_pattern(ignores *ig, const char *pattern);
void load_ignore_patterns(ignores *ig, const char *path);
int filename_filter(const char *path, const struct dirent *dir, void *baton);
int is_empty(ignores *ig);
#endif
================================================
FILE: src/lang.c
================================================
#include <stdlib.h>
#include <string.h>
#include "lang.h"
#include "util.h"
lang_spec_t langs[] = {
{ "actionscript", { "as", "mxml" } },
{ "ada", { "ada", "adb", "ads" } },
{ "asciidoc", { "adoc", "ad", "asc", "asciidoc" } },
{ "apl", { "apl" } },
{ "asm", { "asm", "s" } },
{ "asp", { "asp", "asa", "aspx", "asax", "ashx", "ascx", "asmx" } },
{ "aspx", { "asp", "asa", "aspx", "asax", "ashx", "ascx", "asmx" } },
{ "batch", { "bat", "cmd" } },
{ "bazel", { "bazel" } },
{ "bitbake", { "bb", "bbappend", "bbclass", "inc" } },
{ "cc", { "c", "h", "xs" } },
{ "cfmx", { "cfc", "cfm", "cfml" } },
{ "chpl", { "chpl" } },
{ "clojure", { "clj", "cljs", "cljc", "cljx", "edn" } },
{ "coffee", { "coffee", "cjsx" } },
{ "config", { "config" } },
{ "coq", { "coq", "g", "v" } },
{ "cpp", { "cpp", "cc", "C", "cxx", "m", "hpp", "hh", "h", "H", "hxx", "tpp" } },
{ "crystal", { "cr", "ecr" } },
{ "csharp", { "cs" } },
{ "cshtml", { "cshtml" } },
{ "css", { "css" } },
{ "cython", { "pyx", "pxd", "pxi" } },
{ "delphi", { "pas", "int", "dfm", "nfm", "dof", "dpk", "dpr", "dproj", "groupproj", "bdsgroup", "bdsproj" } },
{ "dlang", { "d", "di" } },
{ "dot", { "dot", "gv" } },
{ "dts", { "dts", "dtsi" } },
{ "ebuild", { "ebuild", "eclass" } },
{ "elisp", { "el" } },
{ "elixir", { "ex", "eex", "exs" } },
{ "elm", { "elm" } },
{ "erlang", { "erl", "hrl" } },
{ "factor", { "factor" } },
{ "fortran", { "f", "F", "f77", "f90", "F90", "f95", "f03", "for", "ftn", "fpp", "FPP" } },
{ "fsharp", { "fs", "fsi", "fsx" } },
{ "gettext", { "po", "pot", "mo" } },
{ "glsl", { "vert", "tesc", "tese", "geom", "frag", "comp" } },
{ "go", { "go" } },
{ "gradle", { "gradle" } },
{ "groovy", { "groovy", "gtmpl", "gpp", "grunit", "gradle" } },
{ "haml", { "haml" } },
{ "handlebars", { "hbs" } },
{ "haskell", { "hs", "hsig", "lhs" } },
{ "haxe", { "hx" } },
{ "hh", { "h" } },
{ "html", { "htm", "html", "shtml", "xhtml" } },
{ "idris", { "idr", "ipkg", "lidr" } },
{ "ini", { "ini" } },
{ "ipython", { "ipynb" } },
{ "isabelle", { "thy" } },
{ "j", { "ijs" } },
{ "jade", { "jade" } },
{ "java", { "java", "properties" } },
{ "jinja2", { "j2" } },
{ "js", { "es6", "js", "jsx", "vue" } },
{ "json", { "json" } },
{ "jsp", { "jsp", "jspx", "jhtm", "jhtml", "jspf", "tag", "tagf" } },
{ "julia", { "jl" } },
{ "kotlin", { "kt" } },
{ "less", { "less" } },
{ "liquid", { "liquid" } },
{ "lisp", { "lisp", "lsp" } },
{ "log", { "log" } },
{ "lua", { "lua" } },
{ "m4", { "m4" } },
{ "make", { "Makefiles", "mk", "mak" } },
{ "mako", { "mako" } },
{ "markdown", { "markdown", "mdown", "mdwn", "mkdn", "mkd", "md" } },
{ "mason", { "mas", "mhtml", "mpl", "mtxt" } },
{ "matlab", { "m" } },
{ "mathematica", { "m", "wl" } },
{ "md", { "markdown", "mdown", "mdwn", "mkdn", "mkd", "md" } },
{ "mercury", { "m", "moo" } },
{ "naccess", { "asa", "rsa" } },
{ "nim", { "nim" } },
{ "nix", { "nix" } },
{ "objc", { "m", "h" } },
{ "objcpp", { "mm", "h" } },
{ "ocaml", { "ml", "mli", "mll", "mly" } },
{ "octave", { "m" } },
{ "org", { "org" } },
{ "parrot", { "pir", "pasm", "pmc", "ops", "pod", "pg", "tg" } },
{ "pdb", { "pdb" } },
{ "perl", { "pl", "pm", "pm6", "pod", "t" } },
{ "php", { "php", "phpt", "php3", "php4", "php5", "phtml" } },
{ "pike", { "pike", "pmod" } },
{ "plist", { "plist" } },
{ "plone", { "pt", "cpt", "metadata", "cpy", "py", "xml", "zcml" } },
{ "powershell", { "ps1" } },
{ "proto", { "proto" } },
{ "ps1", { "ps1" } },
{ "pug", { "pug" } },
{ "puppet", { "pp" } },
{ "python", { "py" } },
{ "qml", { "qml" } },
{ "racket", { "rkt", "ss", "scm" } },
{ "rake", { "Rakefile" } },
{ "razor", { "cshtml" } },
{ "restructuredtext", { "rst" } },
{ "rs", { "rs" } },
{ "r", { "r", "R", "Rmd", "Rnw", "Rtex", "Rrst" } },
{ "rdoc", { "rdoc" } },
{ "ruby", { "rb", "rhtml", "rjs", "rxml", "erb", "rake", "spec" } },
{ "rust", { "rs" } },
{ "salt", { "sls" } },
{ "sass", { "sass", "scss" } },
{ "scala", { "scala" } },
{ "scheme", { "scm", "ss" } },
{ "shell", { "sh", "bash", "csh", "tcsh", "ksh", "zsh", "fish" } },
{ "smalltalk", { "st" } },
{ "sml", { "sml", "fun", "mlb", "sig" } },
{ "sql", { "sql", "ctl" } },
{ "stata", { "do", "ado" } },
{ "stylus", { "styl" } },
{ "swift", { "swift" } },
{ "tcl", { "tcl", "itcl", "itk" } },
{ "terraform", { "tf", "tfvars" } },
{ "tex", { "tex", "cls", "sty" } },
{ "thrift", { "thrift" } },
{ "tla", { "tla" } },
{ "tt", { "tt", "tt2", "ttml" } },
{ "toml", { "toml" } },
{ "ts", { "ts", "tsx" } },
{ "twig", { "twig" } },
{ "vala", { "vala", "vapi" } },
{ "vb", { "bas", "cls", "frm", "ctl", "vb", "resx" } },
{ "velocity", { "vm", "vtl", "vsl" } },
{ "verilog", { "v", "vh", "sv", "svh" } },
{ "vhdl", { "vhd", "vhdl" } },
{ "vim", { "vim" } },
{ "vue", { "vue" } },
{ "wix", { "wxi", "wxs" } },
{ "wsdl", { "wsdl" } },
{ "wadl", { "wadl" } },
{ "xml", { "xml", "dtd", "xsl", "xslt", "xsd", "ent", "tld", "plist", "wsdl" } },
{ "yaml", { "yaml", "yml" } },
{ "zeek", { "zeek", "bro", "bif" } },
{ "zephir", { "zep" } }
};
size_t get_lang_count() {
return sizeof(langs) / sizeof(lang_spec_t);
}
char *make_lang_regex(char *ext_array, size_t num_exts) {
int regex_capacity = 100;
char *regex = ag_malloc(regex_capacity);
int regex_length = 3;
int subsequent = 0;
char *extension;
size_t i;
strcpy(regex, "\\.(");
for (i = 0; i < num_exts; ++i) {
extension = ext_array + i * SINGLE_EXT_LEN;
int extension_length = strlen(extension);
while (regex_length + extension_length + 3 + subsequent > regex_capacity) {
regex_capacity *= 2;
regex = ag_realloc(regex, regex_capacity);
}
if (subsequent) {
regex[regex_length++] = '|';
} else {
subsequent = 1;
}
strcpy(regex + regex_length, extension);
regex_length += extension_length;
}
regex[regex_length++] = ')';
regex[regex_length++] = '$';
regex[regex_length++] = 0;
return regex;
}
size_t combine_file_extensions(size_t *extension_index, size_t len, char **exts) {
/* Keep it fixed as 100 for the reason that if you have more than 100
* file types to search, you'd better search all the files.
* */
size_t ext_capacity = 100;
(*exts) = (char *)ag_malloc(ext_capacity * SINGLE_EXT_LEN);
memset((*exts), 0, ext_capacity * SINGLE_EXT_LEN);
size_t num_of_extensions = 0;
size_t i;
for (i = 0; i < len; ++i) {
size_t j = 0;
const char *ext = langs[extension_index[i]].extensions[j];
do {
if (num_of_extensions == ext_capacity) {
break;
}
char *pos = (*exts) + num_of_extensions * SINGLE_EXT_LEN;
strncpy(pos, ext, strlen(ext));
++num_of_extensions;
ext = langs[extension_index[i]].extensions[++j];
} while (ext);
}
return num_of_extensions;
}
================================================
FILE: src/lang.h
================================================
#ifndef LANG_H
#define LANG_H
#define MAX_EXTENSIONS 12
#define SINGLE_EXT_LEN 20
typedef struct {
const char *name;
const char *extensions[MAX_EXTENSIONS];
} lang_spec_t;
extern lang_spec_t langs[];
/**
Return the language count.
*/
size_t get_lang_count(void);
/**
Convert a NULL-terminated array of language extensions
into a regular expression of the form \.(extension1|extension2...)$
Caller is responsible for freeing the returned string.
*/
char *make_lang_regex(char *ext_array, size_t num_exts);
/**
Combine multiple file type extensions into one array.
The combined result is returned through *exts*;
*exts* is one-dimension array, which can contain up to 100 extensions;
The number of extensions that *exts* actually contain is returned.
*/
size_t combine_file_extensions(size_t *extension_index, size_t len, char **exts);
#endif
================================================
FILE: src/log.c
================================================
#include <stdarg.h>
#include <stdio.h>
#include "log.h"
#include "util.h"
pthread_mutex_t print_mtx = PTHREAD_MUTEX_INITIALIZER;
static enum log_level log_threshold = LOG_LEVEL_ERR;
void set_log_level(enum log_level threshold) {
log_threshold = threshold;
}
void log_debug(const char *fmt, ...) {
va_list args;
va_start(args, fmt);
vplog(LOG_LEVEL_DEBUG, fmt, args);
va_end(args);
}
void log_msg(const char *fmt, ...) {
va_list args;
va_start(args, fmt);
vplog(LOG_LEVEL_MSG, fmt, args);
va_end(args);
}
void log_warn(const char *fmt, ...) {
va_list args;
va_start(args, fmt);
vplog(LOG_LEVEL_WARN, fmt, args);
va_end(args);
}
void log_err(const char *fmt, ...) {
va_list args;
va_start(args, fmt);
vplog(LOG_LEVEL_ERR, fmt, args);
va_end(args);
}
void vplog(const unsigned int level, const char *fmt, va_list args) {
if (level < log_threshold) {
return;
}
pthread_mutex_lock(&print_mtx);
FILE *stream = out_fd;
switch (level) {
case LOG_LEVEL_DEBUG:
fprintf(stream, "DEBUG: ");
break;
case LOG_LEVEL_MSG:
fprintf(stream, "MSG: ");
break;
case LOG_LEVEL_WARN:
fprintf(stream, "WARN: ");
break;
case LOG_LEVEL_ERR:
stream = stderr;
fprintf(stream, "ERR: ");
break;
}
vfprintf(stream, fmt, args);
fprintf(stream, "\n");
pthread_mutex_unlock(&print_mtx);
}
void plog(const unsigned int level, const char *fmt, ...) {
va_list args;
va_start(args, fmt);
vplog(level, fmt, args);
va_end(args);
}
================================================
FILE: src/log.h
================================================
#ifndef LOG_H
#define LOG_H
#include <stdarg.h>
#include "config.h"
#ifdef HAVE_PTHREAD_H
#include <pthread.h>
#endif
extern pthread_mutex_t print_mtx;
enum log_level {
LOG_LEVEL_DEBUG = 10,
LOG_LEVEL_MSG = 20,
LOG_LEVEL_WARN = 30,
LOG_LEVEL_ERR = 40,
LOG_LEVEL_NONE = 100
};
void set_log_level(enum log_level threshold);
void log_debug(const char *fmt, ...);
void log_msg(const char *fmt, ...);
void log_warn(const char *fmt, ...);
void log_err(const char *fmt, ...);
void vplog(const unsigned int level, const char *fmt, va_list args);
void plog(const unsigned int level, const char *fmt, ...);
#endif
================================================
FILE: src/main.c
================================================
#include <ctype.h>
#include <pcre.h>
#include <stdarg.h>
#include <stdio.h>
#include <string.h>
#include <sys/time.h>
#include <unistd.h>
#ifdef _WIN32
#include <windows.h>
#endif
#include "config.h"
#ifdef HAVE_SYS_CPUSET_H
#include <sys/cpuset.h>
#endif
#ifdef HAVE_PTHREAD_H
#include <pthread.h>
#endif
#if defined(HAVE_PTHREAD_SETAFFINITY_NP) && defined(__FreeBSD__)
#include <pthread_np.h>
#endif
#include "log.h"
#include "options.h"
#include "search.h"
#include "util.h"
typedef struct {
pthread_t thread;
int id;
} worker_t;
int main(int argc, char **argv) {
char **base_paths = NULL;
char **paths = NULL;
int i;
int pcre_opts = PCRE_MULTILINE;
int study_opts = 0;
worker_t *workers = NULL;
int workers_len;
int num_cores;
#ifdef HAVE_PLEDGE
if (pledge("stdio rpath proc exec", NULL) == -1) {
die("pledge: %s", strerror(errno));
}
#endif
set_log_level(LOG_LEVEL_WARN);
work_queue = NULL;
work_queue_tail = NULL;
root_ignores = init_ignore(NULL, "", 0);
out_fd = stdout;
parse_options(argc, argv, &base_paths, &paths);
log_debug("PCRE Version: %s", pcre_version());
if (opts.stats) {
memset(&stats, 0, sizeof(stats));
gettimeofday(&(stats.time_start), NULL);
}
#ifdef USE_PCRE_JIT
int has_jit = 0;
pcre_config(PCRE_CONFIG_JIT, &has_jit);
if (has_jit) {
study_opts |= PCRE_STUDY_JIT_COMPILE;
}
#endif
#ifdef _WIN32
{
SYSTEM_INFO si;
GetSystemInfo(&si);
num_cores = si.dwNumberOfProcessors;
}
#else
num_cores = (int)sysconf(_SC_NPROCESSORS_ONLN);
#endif
workers_len = num_cores < 8 ? num_cores : 8;
if (opts.literal) {
workers_len--;
}
if (opts.workers) {
workers_len = opts.workers;
}
if (workers_len < 1) {
workers_len = 1;
}
log_debug("Using %i workers", workers_len);
done_adding_files = FALSE;
workers = ag_calloc(workers_len, sizeof(worker_t));
if (pthread_cond_init(&files_ready, NULL)) {
die("pthread_cond_init failed!");
}
if (pthread_mutex_init(&print_mtx, NULL)) {
die("pthread_mutex_init failed!");
}
if (opts.stats && pthread_mutex_init(&stats_mtx, NULL)) {
die("pthread_mutex_init failed!");
}
if (pthread_mutex_init(&work_queue_mtx, NULL)) {
die("pthread_mutex_init failed!");
}
if (opts.casing == CASE_SMART) {
opts.casing = is_lowercase(opts.query) ? CASE_INSENSITIVE : CASE_SENSITIVE;
}
if (opts.literal) {
if (opts.casing == CASE_INSENSITIVE) {
/* Search routine needs the query to be lowercase */
char *c = opts.query;
for (; *c != '\0'; ++c) {
*c = (char)tolower(*c);
}
}
generate_alpha_skip(opts.query, opts.query_len, alpha_skip_lookup, opts.casing == CASE_SENSITIVE);
find_skip_lookup = NULL;
generate_find_skip(opts.query, opts.query_len, &find_skip_lookup, opts.casing == CASE_SENSITIVE);
generate_hash(opts.query, opts.query_len, h_table, opts.casing == CASE_SENSITIVE);
if (opts.word_regexp) {
init_wordchar_table();
opts.literal_starts_wordchar = is_wordchar(opts.query[0]);
opts.literal_ends_wordchar = is_wordchar(opts.query[opts.query_len - 1]);
}
} else {
if (opts.casing == CASE_INSENSITIVE) {
pcre_opts |= PCRE_CASELESS;
}
if (opts.word_regexp) {
char *word_regexp_query;
ag_asprintf(&word_regexp_query, "\\b(?:%s)\\b", opts.query);
free(opts.query);
opts.query = word_regexp_query;
opts.query_len = strlen(opts.query);
}
compile_study(&opts.re, &opts.re_extra, opts.query, pcre_opts, study_opts);
}
if (opts.search_stream) {
search_stream(stdin, "");
} else {
for (i = 0; i < workers_len; i++) {
workers[i].id = i;
int rv = pthread_create(&(workers[i].thread), NULL, &search_file_worker, &(workers[i].id));
if (rv != 0) {
die("Error in pthread_create(): %s", strerror(rv));
}
#if defined(HAVE_PTHREAD_SETAFFINITY_NP) && (defined(USE_CPU_SET) || defined(HAVE_SYS_CPUSET_H))
if (opts.use_thread_affinity) {
#if defined(__linux__) || defined(__midipix__)
cpu_set_t cpu_set;
#elif __FreeBSD__
cpuset_t cpu_set;
#endif
CPU_ZERO(&cpu_set);
CPU_SET(i % num_cores, &cpu_set);
rv = pthread_setaffinity_np(workers[i].thread, sizeof(cpu_set), &cpu_set);
if (rv) {
log_err("Error in pthread_setaffinity_np(): %s", strerror(rv));
log_err("Performance may be affected. Use --noaffinity to suppress this message.");
} else {
log_debug("Thread %i set to CPU %i", i, i);
}
} else {
log_debug("Thread affinity disabled.");
}
#else
log_debug("No CPU affinity support.");
#endif
}
#ifdef HAVE_PLEDGE
if (pledge("stdio rpath", NULL) == -1) {
die("pledge: %s", strerror(errno));
}
#endif
for (i = 0; paths[i] != NULL; i++) {
log_debug("searching path %s for %s", paths[i], opts.query);
symhash = NULL;
ignores *ig = init_ignore(root_ignores, "", 0);
struct stat s = { .st_dev = 0 };
#ifndef _WIN32
/* The device is ignored if opts.one_dev is false, so it's fine
* to leave it at the default 0
*/
if (opts.one_dev && lstat(paths[i], &s) == -1) {
log_err("Failed to get device information for path %s. Skipping...", paths[i]);
}
#endif
search_dir(ig, base_paths[i], paths[i], 0, s.st_dev);
cleanup_ignore(ig);
}
pthread_mutex_lock(&work_queue_mtx);
done_adding_files = TRUE;
pthread_cond_broadcast(&files_ready);
pthread_mutex_unlock(&work_queue_mtx);
for (i = 0; i < workers_len; i++) {
if (pthread_join(workers[i].thread, NULL)) {
die("pthread_join failed!");
}
}
}
if (opts.stats) {
gettimeofday(&(stats.time_end), NULL);
double time_diff = ((long)stats.time_end.tv_sec * 1000000 + stats.time_end.tv_usec) -
((long)stats.time_start.tv_sec * 1000000 + stats.time_start.tv_usec);
time_diff /= 1000000;
printf("%zu matches\n%zu files contained matches\n%zu files searched\n%zu bytes searched\n%f seconds\n",
stats.total_matches, stats.total_file_matches, stats.total_files, stats.total_bytes, time_diff);
pthread_mutex_destroy(&stats_mtx);
}
if (opts.pager) {
pclose(out_fd);
}
cleanup_options();
pthread_cond_destroy(&files_ready);
pthread_mutex_destroy(&work_queue_mtx);
pthread_mutex_destroy(&print_mtx);
cleanup_ignore(root_ignores);
free(workers);
for (i = 0; paths[i] != NULL; i++) {
free(paths[i]);
free(base_paths[i]);
}
free(base_paths);
free(paths);
if (find_skip_lookup) {
free(find_skip_lookup);
}
return !opts.match_found;
}
================================================
FILE: src/options.c
================================================
#include <errno.h>
#include <limits.h>
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/param.h>
#include <sys/stat.h>
#include <unistd.h>
#include "config.h"
#include "ignore.h"
#include "lang.h"
#include "log.h"
#include "options.h"
#include "print.h"
#include "util.h"
const char *color_line_number = "\033[1;33m"; /* bold yellow */
const char *color_match = "\033[30;43m"; /* black with yellow background */
const char *color_path = "\033[1;32m"; /* bold green */
cli_options opts;
/* TODO: try to obey out_fd? */
void usage(void) {
printf("\n");
printf("Usage: ag [FILE-TYPE] [OPTIONS] PATTERN [PATH]\n\n");
printf(" Recursively search for PATTERN in PATH.\n");
printf(" Like grep or ack, but faster.\n\n");
printf("Example:\n ag -i foo /bar/\n\n");
printf("\
Output Options:\n\
--ackmate Print results in AckMate-parseable format\n\
-A --after [LINES] Print lines after match (Default: 2)\n\
-B --before [LINES] Print lines before match (Default: 2)\n\
--[no]break Print newlines between matches in different files\n\
(Enabled by default)\n\
-c --count Only print the number of matches in each file.\n\
(This often differs from the number of matching lines)\n\
--[no]color Print color codes in results (Enabled by default)\n\
--color-line-number Color codes for line numbers (Default: 1;33)\n\
--color-match Color codes for result match numbers (Default: 30;43)\n\
--color-path Color codes for path names (Default: 1;32)\n\
");
#ifdef _WIN32
printf("\
--color-win-ansi Use ansi colors on Windows even where we can use native\n\
(pager/pipe colors are ansi regardless) (Default: off)\n\
");
#endif
printf("\
--column Print column numbers in results\n\
--[no]filename Print file names (Enabled unless searching a single file)\n\
-H --[no]heading Print file names before each file's matches\n\
(Enabled by default)\n\
-C --context [LINES] Print lines before and after matches (Default: 2)\n\
--[no]group Same as --[no]break --[no]heading\n\
-g --filename-pattern PATTERN\n\
Print filenames matching PATTERN\n\
-l --files-with-matches Only print filenames that contain matches\n\
(don't print the matching lines)\n\
-L --files-without-matches\n\
Only print filenames that don't contain matches\n\
--print-all-files Print headings for all files searched, even those that\n\
don't contain matches\n\
--[no]numbers Print line numbers. Default is to omit line numbers\n\
when searching streams\n\
-o --only-matching Prints only the matching part of the lines\n\
--print-long-lines Print matches on very long lines (Default: >2k characters)\n\
--passthrough When searching a stream, print all lines even if they\n\
don't match\n\
--silent Suppress all log messages, including errors\n\
--stats Print stats (files scanned, time taken, etc.)\n\
--stats-only Print stats and nothing else.\n\
(Same as --count when searching a single file)\n\
--vimgrep Print results like vim's :vimgrep /pattern/g would\n\
(it reports every match on the line)\n\
-0 --null --print0 Separate filenames with null (for 'xargs -0')\n\
\n\
Search Options:\n\
-a --all-types Search all files (doesn't include hidden files\n\
or patterns from ignore files)\n\
-D --debug Ridiculous debugging (probably not useful)\n\
--depth NUM Search up to NUM directories deep (Default: 25)\n\
-f --follow Follow symlinks\n\
-F --fixed-strings Alias for --literal for compatibility with grep\n\
-G --file-search-regex PATTERN Limit search to filenames matching PATTERN\n\
--hidden Search hidden files (obeys .*ignore files)\n\
-i --ignore-case Match case insensitively\n\
--ignore PATTERN Ignore files/directories matching PATTERN\n\
(literal file/directory names also allowed)\n\
--ignore-dir NAME Alias for --ignore for compatibility with ack.\n\
-m --max-count NUM Skip the rest of a file after NUM matches (Default: 10,000)\n\
--one-device Don't follow links to other devices.\n\
-p --path-to-ignore STRING\n\
Use .ignore file at STRING\n\
-Q --literal Don't parse PATTERN as a regular expression\n\
-s --case-sensitive Match case sensitively\n\
-S --smart-case Match case insensitively unless PATTERN contains\n\
uppercase characters (Enabled by default)\n\
--search-binary Search binary files for matches\n\
-t --all-text Search all text files (doesn't include hidden files)\n\
-u --unrestricted Search all files (ignore .ignore, .gitignore, etc.;\n\
searches binary and hidden files as well)\n\
-U --skip-vcs-ignores Ignore VCS ignore files\n\
(.gitignore, .hgignore; still obey .ignore)\n\
-v --invert-match\n\
-w --word-regexp Only match whole words\n\
-W --width NUM Truncate match lines after NUM characters\n\
-z --search-zip Search contents of compressed (e.g., gzip) files\n\
\n");
printf("File Types:\n\
The search can be restricted to certain types of files. Example:\n\
ag --html needle\n\
- Searches for 'needle' in files with suffix .htm, .html, .shtml or .xhtml.\n\
\n\
For a list of supported file types run:\n\
ag --list-file-types\n\n\
ag was originally created by Geoff Greer. More information (and the latest release)\n\
can be found at http://geoff.greer.fm/ag\n");
}
void print_version(void) {
char jit = '-';
char lzma = '-';
char zlib = '-';
#ifdef USE_PCRE_JIT
jit = '+';
#endif
#ifdef HAVE_LZMA_H
lzma = '+';
#endif
#ifdef HAVE_ZLIB_H
zlib = '+';
#endif
printf("ag version %s\n\n", PACKAGE_VERSION);
printf("Features:\n");
printf(" %cjit %clzma %czlib\n", jit, lzma, zlib);
}
void init_options(void) {
char *term = getenv("TERM");
memset(&opts, 0, sizeof(opts));
opts.casing = CASE_DEFAULT;
opts.color = TRUE;
if (term && !strcmp(term, "dumb")) {
opts.color = FALSE;
}
opts.color_win_ansi = FALSE;
opts.max_matches_per_file = 0;
opts.max_search_depth = DEFAULT_MAX_SEARCH_DEPTH;
#if defined(__APPLE__) || defined(__MACH__)
/* mamp() is slower than normal read() on macos. default to off */
opts.mmap = FALSE;
#else
opts.mmap = TRUE;
#endif
opts.multiline = TRUE;
opts.width = 0;
opts.path_sep = '\n';
opts.print_break = TRUE;
opts.print_path = PATH_PRINT_DEFAULT;
opts.print_all_paths = FALSE;
opts.print_line_numbers = TRUE;
opts.recurse_dirs = TRUE;
opts.color_path = ag_strdup(color_path);
opts.color_match = ag_strdup(color_match);
opts.color_line_number = ag_strdup(color_line_number);
opts.use_thread_affinity = TRUE;
}
void cleanup_options(void) {
free(opts.color_path);
free(opts.color_match);
free(opts.color_line_number);
if (opts.query) {
free(opts.query);
}
pcre_free(opts.re);
if (opts.re_extra) {
/* Using pcre_free_study on pcre_extra* can segfault on some versions of PCRE */
pcre_free(opts.re_extra);
}
if (opts.ackmate_dir_filter) {
pcre_free(opts.ackmate_dir_filter);
}
if (opts.ackmate_dir_filter_extra) {
pcre_free(opts.ackmate_dir_filter_extra);
}
if (opts.file_search_regex) {
pcre_free(opts.file_search_regex);
}
if (opts.file_search_regex_extra) {
pcre_free(opts.file_search_regex_extra);
}
}
void parse_options(int argc, char **argv, char **base_paths[], char **paths[]) {
int ch;
size_t i;
int path_len = 0;
int base_path_len = 0;
int useless = 0;
int group = 1;
int help = 0;
int version = 0;
int list_file_types = 0;
int opt_index = 0;
char *num_end;
const char *home_dir = getenv("HOME");
char *ignore_file_path = NULL;
int accepts_query = 1;
int needs_query = 1;
struct stat statbuf;
int rv;
size_t lang_count;
size_t lang_num = 0;
int has_filetype = 0;
size_t longopts_len, full_len;
option_t *longopts;
char *lang_regex = NULL;
size_t *ext_index = NULL;
char *extensions = NULL;
size_t num_exts = 0;
init_options();
option_t base_longopts[] = {
{ "ackmate", no_argument, &opts.ackmate, 1 },
{ "ackmate-dir-filter", required_argument, NULL, 0 },
{ "affinity", no_argument, &opts.use_thread_affinity, 1 },
{ "after", optional_argument, NULL, 'A' },
{ "all-text", no_argument, NULL, 't' },
{ "all-types", no_argument, NULL, 'a' },
{ "before", optional_argument, NULL, 'B' },
{ "break", no_argument, &opts.print_break, 1 },
{ "case-sensitive", no_argument, NULL, 's' },
{ "color", no_argument, &opts.color, 1 },
{ "color-line-number", required_argument, NULL, 0 },
{ "color-match", required_argument, NULL, 0 },
{ "color-path", required_argument, NULL, 0 },
{ "color-win-ansi", no_argument, &opts.color_win_ansi, TRUE },
{ "column", no_argument, &opts.column, 1 },
{ "context", optional_argument, NULL, 'C' },
{ "count", no_argument, NULL, 'c' },
{ "debug", no_argument, NULL, 'D' },
{ "depth", required_argument, NULL, 0 },
{ "filename", no_argument, NULL, 0 },
{ "filename-pattern", required_argument, NULL, 'g' },
{ "file-search-regex", required_argument, NULL, 'G' },
{ "files-with-matches", no_argument, NULL, 'l' },
{ "files-without-matches", no_argument, NULL, 'L' },
{ "fixed-strings", no_argument, NULL, 'F' },
{ "follow", no_argument, &opts.follow_symlinks, 1 },
{ "group", no_argument, &group, 1 },
{ "heading", no_argument, &opts.print_path, PATH_PRINT_TOP },
{ "help", no_argument, NULL, 'h' },
{ "hidden", no_argument, &opts.search_hidden_files, 1 },
{ "ignore", required_argument, NULL, 0 },
{ "ignore-case", no_argument, NULL, 'i' },
{ "ignore-dir", required_argument, NULL, 0 },
{ "invert-match", no_argument, NULL, 'v' },
/* deprecated for --numbers. Remove eventually. */
{ "line-numbers", no_argument, &opts.print_line_numbers, 2 },
{ "list-file-types", no_argument, &list_file_types, 1 },
{ "literal", no_argument, NULL, 'Q' },
{ "match", no_argument, &useless, 0 },
{ "max-count", required_argument, NULL, 'm' },
{ "mmap", no_argument, &opts.mmap, TRUE },
{ "multiline", no_argument, &opts.multiline, TRUE },
/* Accept both --no-* and --no* forms for convenience/BC */
{ "no-affinity", no_argument, &opts.use_thread_affinity, 0 },
{ "noaffinity", no_argument, &opts.use_thread_affinity, 0 },
{ "no-break", no_argument, &opts.print_break, 0 },
{ "nobreak", no_argument, &opts.print_break, 0 },
{ "no-color", no_argument, &opts.color, 0 },
{ "nocolor", no_argument, &opts.color, 0 },
{ "no-filename", no_argument, NULL, 0 },
{ "nofilename", no_argument, NULL, 0 },
{ "no-follow", no_argument, &opts.follow_symlinks, 0 },
{ "nofollow", no_argument, &opts.follow_symlinks, 0 },
{ "no-group", no_argument, &group, 0 },
{ "nogroup", no_argument, &group, 0 },
{ "no-heading", no_argument, &opts.print_path, PATH_PRINT_EACH_LINE },
{ "noheading", no_argument, &opts.print_path, PATH_PRINT_EACH_LINE },
{ "no-mmap", no_argument, &opts.mmap, FALSE },
{ "nommap", no_argument, &opts.mmap, FALSE },
{ "no-multiline", no_argument, &opts.multiline, FALSE },
{ "nomultiline", no_argument, &opts.multiline, FALSE },
{ "no-numbers", no_argument, &opts.print_line_numbers, FALSE },
{ "nonumbers", no_argument, &opts.print_line_numbers, FALSE },
{ "no-pager", no_argument, NULL, 0 },
{ "nopager", no_argument, NULL, 0 },
{ "no-recurse", no_argument, NULL, 'n' },
{ "norecurse", no_argument, NULL, 'n' },
{ "null", no_argument, NULL, '0' },
{ "numbers", no_argument, &opts.print_line_numbers, 2 },
{ "only-matching", no_argument, NULL, 'o' },
{ "one-device", no_argument, &opts.one_dev, 1 },
{ "pager", required_argument, NULL, 0 },
{ "parallel", no_argument, &opts.parallel, 1 },
{ "passthrough", no_argument, &opts.passthrough, 1 },
{ "passthru", no_argument, &opts.passthrough, 1 },
{ "path-to-ignore", required_argument, NULL, 'p' },
{ "print0", no_argument, NULL, '0' },
{ "print-all-files", no_argument, NULL, 0 },
{ "print-long-lines", no_argument, &opts.print_long_lines, 1 },
{ "recurse", no_argument, NULL, 'r' },
{ "search-binary", no_argument, &opts.search_binary_files, 1 },
{ "search-files", no_argument, &opts.search_stream, 0 },
{ "search-zip", no_argument, &opts.search_zip_files, 1 },
{ "silent", no_argument, NULL, 0 },
{ "skip-vcs-ignores", no_argument, NULL, 'U' },
{ "smart-case", no_argument, NULL, 'S' },
{ "stats", no_argument, &opts.stats, 1 },
{ "stats-only", no_argument, NULL, 0 },
{ "unrestricted", no_argument, NULL, 'u' },
{ "version", no_argument, &version, 1 },
{ "vimgrep", no_argument, &opts.vimgrep, 1 },
{ "width", required_argument, NULL, 'W' },
{ "word-regexp", no_argument, NULL, 'w' },
{ "workers", required_argument, NULL, 0 },
};
lang_count = get_lang_count();
longopts_len = (sizeof(base_longopts) / sizeof(option_t));
full_len = (longopts_len + lang_count + 1);
longopts = ag_malloc(full_len * sizeof(option_t));
memcpy(longopts, base_longopts, sizeof(base_longopts));
ext_index = (size_t *)ag_malloc(sizeof(size_t) * lang_count);
memset(ext_index, 0, sizeof(size_t) * lang_count);
for (i = 0; i < lang_count; i++) {
option_t opt = { langs[i].name, no_argument, NULL, 0 };
longopts[i + longopts_len] = opt;
}
longopts[full_len - 1] = (option_t){ NULL, 0, NULL, 0 };
if (argc < 2) {
usage();
cleanup_ignore(root_ignores);
cleanup_options();
exit(1);
}
rv = fstat(fileno(stdin), &statbuf);
if (rv == 0) {
if (S_ISFIFO(statbuf.st_mode) || S_ISREG(statbuf.st_mode)) {
opts.search_stream = 1;
}
}
/* If we're not outputting to a terminal. change output to:
* turn off colors
* print filenames on every line
*/
if (!isatty(fileno(stdout))) {
opts.color = 0;
group = 0;
/* Don't search the file that stdout is redirected to */
rv = fstat(fileno(stdout), &statbuf);
if (rv != 0) {
die("Error fstat()ing stdout");
}
opts.stdout_inode = statbuf.st_ino;
}
char *file_search_regex = NULL;
while ((ch = getopt_long(argc, argv, "A:aB:C:cDG:g:FfHhiLlm:nop:QRrSsvVtuUwW:z0", longopts, &opt_index)) != -1) {
switch (ch) {
case 'A':
if (optarg) {
opts.after = strtol(optarg, &num_end, 10);
if (num_end == optarg || *num_end != '\0' || errno == ERANGE) {
/* This arg must be the search string instead of the after length */
optind--;
opts.after = DEFAULT_AFTER_LEN;
}
} else {
opts.after = DEFAULT_AFTER_LEN;
}
break;
case 'a':
opts.search_all_files = 1;
opts.search_binary_files = 1;
break;
case 'B':
if (optarg) {
opts.before = strtol(optarg, &num_end, 10);
if (num_end == optarg || *num_end != '\0' || errno == ERANGE) {
/* This arg must be the search string instead of the before length */
optind--;
opts.before = DEFAULT_BEFORE_LEN;
}
} else {
opts.before = DEFAULT_BEFORE_LEN;
}
break;
case 'C':
if (optarg) {
opts.context = strtol(optarg, &num_end, 10);
if (num_end == optarg || *num_end != '\0' || errno == ERANGE) {
/* This arg must be the search string instead of the context length */
optind--;
opts.context = DEFAULT_CONTEXT_LEN;
}
} else {
opts.context = DEFAULT_CONTEXT_LEN;
}
break;
case 'c':
opts.print_count = 1;
opts.print_filename_only = 1;
break;
case 'D':
set_log_level(LOG_LEVEL_DEBUG);
break;
case 'f':
opts.follow_symlinks = 1;
break;
case 'g':
needs_query = accepts_query = 0;
opts.match_files = 1;
/* fall through */
case 'G':
if (file_search_regex) {
log_err("File search regex (-g or -G) already specified.");
usage();
exit(1);
}
file_search_regex = ag_strdup(optarg);
break;
case 'H':
opts.print_path = PATH_PRINT_TOP;
break;
case 'h':
help = 1;
break;
case 'i':
opts.casing = CASE_INSENSITIVE;
break;
case 'L':
opts.print_nonmatching_files = 1;
opts.print_path = PATH_PRINT_TOP;
break;
case 'l':
needs_query = 0;
opts.print_filename_only = 1;
opts.print_path = PATH_PRINT_TOP;
break;
case 'm':
opts.max_matches_per_file = atoi(optarg);
break;
case 'n':
opts.recurse_dirs = 0;
break;
case 'p':
opts.path_to_ignore = TRUE;
load_ignore_patterns(root_ignores, optarg);
break;
case 'o':
opts.only_matching = 1;
break;
case 'F':
case 'Q':
opts.literal = 1;
break;
case 'R':
case 'r':
opts.recurse_dirs = 1;
break;
case 'S':
opts.casing = CASE_SMART;
break;
case 's':
opts.casing = CASE_SENSITIVE;
break;
case 't':
opts.search_all_files = 1;
break;
case 'u':
opts.search_binary_files = 1;
opts.search_all_files = 1;
opts.search_hidden_files = 1;
break;
case 'U':
opts.skip_vcs_ignores = 1;
break;
case 'v':
opts.invert_match = 1;
/* Color highlighting doesn't make sense when inverting matches */
opts.color = 0;
break;
case 'V':
version = 1;
break;
case 'w':
opts.word_regexp = 1;
break;
case 'W':
opts.width = strtol(optarg, &num_end, 10);
if (num_end == optarg || *num_end != '\0' || errno == ERANGE) {
die("Invalid width\n");
}
break;
case 'z':
opts.search_zip_files = 1;
break;
case '0':
opts.path_sep = '\0';
break;
case 0: /* Long option */
if (strcmp(longopts[opt_index].name, "ackmate-dir-filter") == 0) {
compile_study(&opts.ackmate_dir_filter, &opts.ackmate_dir_filter_extra, optarg, 0, 0);
break;
} else if (strcmp(longopts[opt_index].name, "depth") == 0) {
opts.max_search_depth = atoi(optarg);
break;
} else if (strcmp(longopts[opt_index].name, "filename") == 0) {
opts.print_path = PATH_PRINT_DEFAULT;
opts.print_line_numbers = TRUE;
break;
} else if (strcmp(longopts[opt_index].name, "ignore-dir") == 0) {
add_ignore_pattern(root_ignores, optarg);
break;
} else if (strcmp(longopts[opt_index].name, "ignore") == 0) {
add_ignore_pattern(root_ignores, optarg);
break;
} else if (strcmp(longopts[opt_index].name, "no-filename") == 0 ||
strcmp(longopts[opt_index].name, "nofilename") == 0) {
opts.print_path = PATH_PRINT_NOTHING;
opts.print_line_numbers = FALSE;
break;
} else if (strcmp(longopts[opt_index].name, "no-pager") == 0 ||
strcmp(longopts[opt_index].name, "nopager") == 0) {
out_fd = stdout;
opts.pager = NULL;
break;
} else if (strcmp(longopts[opt_index].name, "pager") == 0) {
opts.pager = optarg;
break;
} else if (strcmp(longopts[opt_index].name, "print-all-files") == 0) {
opts.print_all_paths = TRUE;
break;
} else if (strcmp(longopts[opt_index].name, "workers") == 0) {
opts.workers = atoi(optarg);
break;
} else if (strcmp(longopts[opt_index].name, "color-line-number") == 0) {
free(opts.color_line_number);
ag_asprintf(&opts.color_line_number, "\033[%sm", optarg);
break;
} else if (strcmp(longopts[opt_index].name, "color-match") == 0) {
free(opts.color_match);
ag_asprintf(&opts.color_match, "\033[%sm", optarg);
break;
} else if (strcmp(longopts[opt_index].name, "color-path") == 0) {
free(opts.color_path);
ag_asprintf(&opts.color_path, "\033[%sm", optarg);
break;
} else if (strcmp(longopts[opt_index].name, "silent") == 0) {
set_log_level(LOG_LEVEL_NONE);
break;
} else if (strcmp(longopts[opt_index].name, "stats-only") == 0) {
opts.print_filename_only = 1;
opts.print_path = PATH_PRINT_NOTHING;
opts.stats = 1;
break;
}
/* Continue to usage if we don't recognize the option */
if (longopts[opt_index].flag != 0) {
break;
}
for (i = 0; i < lang_count; i++) {
if (strcmp(longopts[opt_index].name, langs[i].name) == 0) {
has_filetype = 1;
ext_index[lang_num++] = i;
break;
}
}
if (i != lang_count) {
break;
}
log_err("option %s does not take a value", longopts[opt_index].name);
/* fall through */
default:
usage();
exit(1);
}
}
if (opts.casing == CASE_DEFAULT) {
opts.casing = CASE_SMART;
}
if (file_search_regex) {
int pcre_opts = 0;
if (opts.casing == CASE_INSENSITIVE || (opts.casing == CASE_SMART && is_lowercase(file_search_regex))) {
pcre_opts |= PCRE_CASELESS;
}
if (opts.word_regexp) {
char *old_file_search_regex = file_search_regex;
ag_asprintf(&file_search_regex, "\\b%s\\b", file_search_regex);
free(old_file_search_regex);
}
compile_study(&opts.file_search_regex, &opts.file_search_regex_extra, file_search_regex, pcre_opts, 0);
free(file_search_regex);
}
if (has_filetype) {
num_exts = combine_file_extensions(ext_index, lang_num, &extensions);
lang_regex = make_lang_regex(extensions, num_exts);
compile_study(&opts.file_search_regex, &opts.file_search_regex_extra, lang_regex, 0, 0);
}
if (extensions) {
free(extensions);
}
free(ext_index);
if (lang_regex) {
free(lang_regex);
}
free(longopts);
argc -= optind;
argv += optind;
if (opts.pager) {
out_fd = popen(opts.pager, "w");
if (!out_fd) {
perror("Failed to run pager");
exit(1);
}
}
#ifdef HAVE_PLEDGE
if (opts.skip_vcs_ignores) {
if (pledge("stdio rpath proc", NULL) == -1) {
die("pledge: %s", strerror(errno));
}
}
#endif
if (help) {
usage();
exit(0);
}
if (version) {
print_version();
exit(0);
}
if (list_file_types) {
size_t lang_index;
printf("The following file types are supported:\n");
for (lang_index = 0; lang_index < lang_count; lang_index++) {
printf(" --%s\n ", langs[lang_index].name);
int j;
for (j = 0; j < MAX_EXTENSIONS && langs[lang_index].extensions[j]; j++) {
printf(" .%s", langs[lang_index].extensions[j]);
}
printf("\n\n");
}
exit(0);
}
if (needs_query && argc == 0) {
log_err("What do you want to search for?");
exit(1);
}
if (home_dir && !opts.search_all_files) {
log_debug("Found user's home dir: %s", home_dir);
ag_asprintf(&ignore_file_path, "%s/.agignore", home_dir);
load_ignore_patterns(root_ignores, ignore_file_path);
free(ignore_file_path);
}
if (!opts.skip_vcs_ignores) {
FILE *gitconfig_file = NULL;
size_t buf_len = 0;
char *gitconfig_res = NULL;
#ifdef _WIN32
gitconfig_file = popen("git config -z --path --get core.excludesfile 2>NUL", "r");
#else
gitconfig_file = popen("git config -z --path --get core.excludesfile 2>/dev/null", "r");
#endif
if (gitconfig_file != NULL) {
do {
gitconfig_res = ag_realloc(gitconfig_res, buf_len + 65);
buf_len += fread(gitconfig_res + buf_len, 1, 64, gitconfig_file);
} while (!feof(gitconfig_file) && buf_len > 0 && buf_len % 64 == 0);
gitconfig_res[buf_len] = '\0';
if (buf_len == 0) {
free(gitconfig_res);
const char *config_home = getenv("XDG_CONFIG_HOME");
if (config_home) {
ag_asprintf(&gitconfig_res, "%s/%s", config_home, "git/ignore");
} else if (home_dir) {
ag_asprintf(&gitconfig_res, "%s/%s", home_dir, ".config/git/ignore");
} else {
gitconfig_res = ag_strdup("");
}
}
log_debug("global core.excludesfile: %s", gitconfig_res);
load_ignore_patterns(root_ignores, gitconfig_res);
free(gitconfig_res);
pclose(gitconfig_file);
}
}
#ifdef HAVE_PLEDGE
if (pledge("stdio rpath proc", NULL) == -1) {
die("pledge: %s", strerror(errno));
}
#endif
if (opts.context > 0) {
opts.before = opts.context;
opts.after = opts.context;
}
if (opts.ackmate) {
opts.color = 0;
opts.print_break = 1;
group = 1;
opts.search_stream = 0;
}
if (opts.vimgrep) {
opts.color = 0;
opts.print_break = 0;
group = 1;
opts.search_stream = 0;
opts.print_path = PATH_PRINT_NOTHING;
}
if (opts.parallel) {
opts.search_stream = 0;
}
if (!(opts.print_path != PATH_PRINT_DEFAULT || opts.print_break == 0)) {
if (group) {
opts.print_break = 1;
} else {
opts.print_path = PATH_PRINT_DEFAULT_EACH_LINE;
opts.print_break = 0;
}
}
if (opts.search_stream) {
opts.print_break = 0;
opts.print_path = PATH_PRINT_NOTHING;
if (opts.print_line_numbers != 2) {
opts.print_line_numbers = 0;
}
}
if (accepts_query && argc > 0) {
if (!needs_query && strlen(argv[0]) == 0) {
// use default query
opts.query = ag_strdup(".");
} else {
// use the provided query
opts.query = ag_strdup(argv[0]);
}
argc--;
argv++;
} else if (!needs_query) {
// use default query
opts.query = ag_strdup(".");
}
opts.query_len = strlen(opts.query);
log_debug("Query is %s", opts.query);
if (opts.query_len == 0) {
log_err("Error: No query. What do you want to search for?");
exit(1);
}
if (!is_regex(opts.query)) {
opts.literal = 1;
}
char *path = NULL;
char *base_path = NULL;
#ifdef PATH_MAX
char *tmp = NULL;
#endif
opts.paths_len = argc;
if (argc > 0) {
*paths = ag_calloc(sizeof(char *), argc + 1);
*base_paths = ag_calloc(sizeof(char *), argc + 1);
for (i = 0; i < (size_t)argc; i++) {
path = ag_strdup(argv[i]);
path_len = strlen(path);
/* kill trailing slash */
if (path_len > 1 && path[path_len - 1] == '/') {
path[path_len - 1] = '\0';
}
(*paths)[i] = path;
#ifdef PATH_MAX
tmp = ag_malloc(PATH_MAX);
base_path = realpath(path, tmp);
#else
base_path = realpath(path, NULL);
#endif
if (base_path) {
base_path_len = strlen(base_path);
/* add trailing slash */
if (base_path_len > 1 && base_path[base_path_len - 1] != '/') {
base_path = ag_realloc(base_path, base_path_len + 2);
base_path[base_path_len] = '/';
base_path[base_path_len + 1] = '\0';
}
}
(*base_paths)[i] = base_path;
}
/* Make sure we search these paths instead of stdin. */
opts.search_stream = 0;
} else {
path = ag_strdup(".");
*paths = ag_malloc(sizeof(char *) * 2);
*base_paths = ag_malloc(sizeof(char *) * 2);
(*paths)[0] = path;
#ifdef PATH_MAX
tmp = ag_malloc(PATH_MAX);
(*base_paths)[0] = realpath(path, tmp);
#else
(*base_paths)[0] = realpath(path, NULL);
#endif
i = 1;
}
(*paths)[i] = NULL;
(*base_paths)[i] = NULL;
#ifdef _WIN32
windows_use_ansi(opts.color_win_ansi);
#endif
}
================================================
FILE: src/options.h
================================================
#ifndef OPTIONS_H
#define OPTIONS_H
#include <getopt.h>
#include <sys/stat.h>
#include <pcre.h>
#define DEFAULT_AFTER_LEN 2
#define DEFAULT_BEFORE_LEN 2
#define DEFAULT_CONTEXT_LEN 2
#define DEFAULT_MAX_SEARCH_DEPTH 25
enum case_behavior {
CASE_DEFAULT, /* Changes to CASE_SMART at the end of option parsing */
CASE_SENSITIVE,
CASE_INSENSITIVE,
CASE_SMART,
CASE_SENSITIVE_RETRY_INSENSITIVE /* for future use */
};
enum path_print_behavior {
PATH_PRINT_DEFAULT, /* PRINT_TOP if > 1 file being searched, else PRINT_NOTHING */
PATH_PRINT_DEFAULT_EACH_LINE, /* PRINT_EACH_LINE if > 1 file being searched, else PRINT_NOTHING */
PATH_PRINT_TOP,
PATH_PRINT_EACH_LINE,
PATH_PRINT_NOTHING
};
typedef struct {
int ackmate;
pcre *ackmate_dir_filter;
pcre_extra *ackmate_dir_filter_extra;
size_t after;
size_t before;
enum case_behavior casing;
const char *file_search_string;
int match_files;
pcre *file_search_regex;
pcre_extra *file_search_regex_extra;
int color;
char *color_line_number;
char *color_match;
char *color_path;
int color_win_ansi;
int column;
int context;
int follow_symlinks;
int invert_match;
int literal;
int literal_starts_wordchar;
int literal_ends_wordchar;
size_t max_matches_per_file;
int max_search_depth;
int mmap;
int multiline;
int one_dev;
int only_matching;
char path_sep;
int path_to_ignore;
int print_break;
int print_count;
int print_filename_only;
int print_nonmatching_files;
int print_path;
int print_all_paths;
int print_line_numbers;
int print_long_lines; /* TODO: support this in print.c */
int passthrough;
pcre *re;
pcre_extra *re_extra;
int recurse_dirs;
int search_all_files;
int skip_vcs_ignores;
int search_binary_files;
int search_zip_files;
int search_hidden_files;
int search_stream; /* true if tail -F blah | ag */
int stats;
size_t stream_line_num; /* This should totally not be in here */
int match_found; /* This should totally not be in here */
ino_t stdout_inode;
char *query;
int query_len;
char *pager;
int paths_len;
int parallel;
int use_thread_affinity;
int vimgrep;
size_t width;
int word_regexp;
int workers;
} cli_options;
/* global options. parse_options gives it sane values, everything else reads from it */
extern cli_options opts;
typedef struct option option_t;
void usage(void);
void print_version(void);
void init_options(void);
void parse_options(int argc, char **argv, char **base_paths[], char **paths[]);
void cleanup_options(void);
#endif
================================================
FILE: src/print.c
================================================
#include <limits.h>
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "ignore.h"
#include "log.h"
#include "options.h"
#include "print.h"
#include "search.h"
#include "util.h"
#ifdef _WIN32
#define fprintf(...) fprintf_w32(__VA_ARGS__)
#endif
int first_file_match = 1;
const char *color_reset = "\033[0m\033[K";
const char *truncate_marker = " [...]";
__thread struct print_context {
size_t line;
char **context_prev_lines;
size_t prev_line;
size_t last_prev_line;
size_t prev_line_offset;
size_t line_preceding_current_match_offset;
size_t lines_since_last_match;
size_t last_printed_match;
int in_a_match;
int printing_a_match;
} print_context;
void print_init_context(void) {
if (print_context.context_prev_lines != NULL) {
return;
}
print_context.context_prev_lines = ag_calloc(sizeof(char *), (opts.before + 1));
print_context.line = 1;
print_context.prev_line = 0;
print_context.last_prev_line = 0;
print_context.prev_line_offset = 0;
print_context.line_preceding_current_match_offset = 0;
print_context.lines_since_last_match = INT_MAX;
print_context.last_printed_match = 0;
print_context.in_a_match = FALSE;
print_context.printing_a_match = FALSE;
}
void print_cleanup_context(void) {
size_t i;
if (print_context.context_prev_lines == NULL) {
return;
}
for (i = 0; i < opts.before; i++) {
if (print_context.context_prev_lines[i] != NULL) {
free(print_context.context_prev_lines[i]);
}
}
free(print_context.context_prev_lines);
print_context.context_prev_lines = NULL;
}
void print_context_append(const char *line, size_t len) {
if (opts.before == 0) {
return;
}
if (print_context.context_prev_lines[print_context.last_prev_line] != NULL) {
free(print_context.context_prev_lines[print_context.last_prev_line]);
}
print_context.context_prev_lines[print_context.last_prev_line] = ag_strndup(line, len);
print_context.last_prev_line = (print_context.last_prev_line + 1) % opts.before;
}
void print_trailing_context(const char *path, const char *buf, size_t n) {
char sep = '-';
if (opts.ackmate || opts.vimgrep) {
sep = ':';
}
if (print_context.lines_since_last_match != 0 &&
print_context.lines_since_last_match <= opts.after) {
if (opts.print_path == PATH_PRINT_EACH_LINE) {
print_path(path, ':');
}
print_line_number(print_context.line, sep);
fwrite(buf, 1, n, out_fd);
fputc('\n', out_fd);
}
print_context.line++;
if (!print_context.in_a_match && print_context.lines_since_last_match < INT_MAX) {
print_context.lines_since_last_match++;
}
}
void print_path(const char *path, const char sep) {
if (opts.print_path == PATH_PRINT_NOTHING && !opts.vimgrep) {
return;
}
path = normalize_path(path);
if (opts.ackmate) {
fprintf(out_fd, ":%s%c", path, sep);
} else if (opts.vimgrep) {
fprintf(out_fd, "%s%c", path, sep);
} else {
if (opts.color) {
fprintf(out_fd, "%s%s%s%c", opts.color_path, path, color_reset, sep);
} else {
fprintf(out_fd, "%s%c", path, sep);
}
}
}
void print_path_count(const char *path, const char sep, const size_t count) {
if (*path) {
print_path(path, ':');
}
if (opts.color) {
fprintf(out_fd, "%s%lu%s%c", opts.color_line_number, (unsigned long)count, color_reset, sep);
} else {
fprintf(out_fd, "%lu%c", (unsigned long)count, sep);
}
}
void print_line(const char *buf, size_t buf_pos, size_t prev_line_offset) {
size_t write_chars = buf_pos - prev_line_offset + 1;
if (opts.width > 0 && opts.width < write_chars) {
write_chars = opts.width;
}
fwrite(buf + prev_line_offset, 1, write_chars, out_fd);
}
void print_binary_file_matches(const char *path) {
path = normalize_path(path);
print_file_separator();
fprintf(out_fd, "Binary file %s matches.\n", path);
}
void print_file_matches(const char *path, const char *buf, const size_t buf_len, const match_t matches[], const size_t matches_len) {
size_t cur_match = 0;
ssize_t lines_to_print = 0;
char sep = '-';
size_t i, j;
int blanks_between_matches = opts.context || opts.after || opts.before;
if (opts.ackmate || opts.vimgrep) {
sep = ':';
}
print_file_separator();
if (opts.print_path == PATH_PRINT_DEFAULT) {
opts.print_path = PATH_PRINT_TOP;
} else if (opts.print_path == PATH_PRINT_DEFAULT_EACH_LINE) {
opts.print_path = PATH_PRINT_EACH_LINE;
}
if (opts.print_path == PATH_PRINT_TOP) {
if (opts.print_count) {
print_path_count(path, opts.path_sep, matches_len);
} else {
print_path(path, opts.path_sep);
}
}
for (i = 0; i <= buf_len && (cur_match < matches_len || print_context.lines_since_last_match <= opts.after); i++) {
if (cur_match < matches_len && i == matches[cur_match].start) {
print_context.in_a_match = TRUE;
/* We found the start of a match */
if (cur_match > 0 && blanks_between_matches && print_context.lines_since_last_match > (opts.before + opts.after + 1)) {
fprintf(out_fd, "--\n");
}
if (print_context.lines_since_last_match > 0 && opts.before > 0) {
/* TODO: better, but still needs work */
/* print the previous line(s) */
lines_to_print = print_context.lines_since_last_match - (opts.after + 1);
if (lines_to_print < 0) {
lines_to_print = 0;
} else if ((size_t)lines_to_print > opts.before) {
lines_to_print = opts.before;
}
for (j = (opts.before - lines_to_print); j < opts.before; j++) {
print_context.prev_line = (print_context.last_prev_line + j) % opts.before;
if (print_context.context_prev_lines[print_context.prev_line] != NULL) {
if (opts.print_path == PATH_PRINT_EACH_LINE) {
print_path(path, ':');
}
print_line_number(print_context.line - (opts.before - j), sep);
fprintf(out_fd, "%s\n", print_context.context_prev_lines[print_context.prev_line]);
}
}
}
print_context.lines_since_last_match = 0;
}
if (cur_match < matches_len && i == matches[cur_match].end) {
/* We found the end of a match. */
cur_match++;
print_context.in_a_match = FALSE;
}
/* We found the end of a line. */
if ((i == buf_len || buf[i] == '\n') && opts.before > 0) {
/* We don't want to strcpy the \n */
print_context_append(&buf[print_context.prev_line_offset], i - print_context.prev_line_offset);
}
if (i == buf_len || buf[i] == '\n') {
if (print_context.lines_since_last_match == 0) {
if (opts.print_path == PATH_PRINT_EACH_LINE && !opts.search_stream) {
print_path(path, ':');
}
if (opts.ackmate) {
/* print headers for ackmate to parse */
print_line_number(print_context.line, ';');
for (; print_context.last_printed_match < cur_match; print_context.last_printed_match++) {
size_t start = matches[print_context.last_printed_match].start - print_context.line_preceding_current_match_offset;
fprintf(out_fd, "%lu %lu",
start,
matches[print_context.last_printed_match].end - matches[print_context.last_printed_match].start);
print_context.last_printed_match == cur_match - 1 ? fputc(':', out_fd) : fputc(',', out_fd);
}
print_line(buf, i, print_context.prev_line_offset);
} else if (opts.vimgrep) {
for (; print_context.last_printed_match < cur_match; print_context.last_printed_match++) {
print_path(path, sep);
print_line_number(print_context.line, sep);
print_column_number(matches, print_context.last_printed_match, print_context.prev_line_offset, sep);
print_line(buf, i, print_context.prev_line_offset);
}
} else {
print_line_number(print_context.line, ':');
int printed_match = FALSE;
if (opts.column) {
print_column_number(matches, print_context.last_printed_match, print_context.prev_line_offset, ':');
}
if (print_context.printing_a_match && opts.color) {
fprintf(out_fd, "%s", opts.color_match);
}
for (j = print_context.prev_line_offset; j <= i; j++) {
/* close highlight of match term */
if (print_context.last_printed_match < matches_len && j == matches[print_context.last_printed_match].end) {
if (opts.color) {
fprintf(out_fd, "%s", color_reset);
}
print_context.printing_a_match = FALSE;
print_context.last_printed_match++;
printed_match = TRUE;
if (opts.only_matching) {
fputc('\n', out_fd);
}
}
/* skip remaining characters if truncation width exceeded, needs to be done
* before highlight opening */
if (j < buf_len && opts.width > 0 && j - print_context.prev_line_offset >= opts.width) {
if (j < i) {
fputs(truncate_marker, out_fd);
}
fputc('\n', out_fd);
/* prevent any more characters or highlights */
j = i;
print_context.last_printed_match = matches_len;
}
/* open highlight of match term */
if (print_context.last_printed_match < matches_len && j == matches[print_context.last_printed_match].start) {
if (opts.only_matching && printed_match) {
if (opts.print_path == PATH_PRINT_EACH_LINE) {
print_path(path, ':');
}
print_line_number(print_context.line, ':');
if (opts.column) {
print_column_number(matches, print_context.last_printed_match, print_context.prev_line_offset, ':');
}
}
if (opts.color) {
fprintf(out_fd, "%s", opts.color_match);
}
print_context.printing_a_match = TRUE;
}
/* Don't print the null terminator */
if (j < buf_len) {
/* if only_matching is set, print only matches and newlines */
if (!opts.only_matching || print_context.printing_a_match) {
if (opts.width == 0 || j - print_context.prev_line_offset < opts.width) {
fputc(buf[j], out_fd);
}
}
}
}
if (print_context.printing_a_match && opts.color) {
fprintf(out_fd, "%s", color_reset);
}
}
}
if (opts.search_stream) {
print_context.last_printed_match = 0;
break;
}
/* print context after matching line */
print_trailing_context(path, &buf[print_context.prev_line_offset], i - print_context.prev_line_offset);
print_context.prev_line_offset = i + 1; /* skip the newline */
if (!print_context.in_a_match) {
print_context.line_preceding_current_match_offset = i + 1;
}
/* File doesn't end with a newline. Print one so the output is pretty. */
if (i == buf_len && buf[i - 1] != '\n') {
fputc('\n', out_fd);
}
}
}
/* Flush output if stdout is not a tty */
if (opts.stdout_inode) {
fflush(out_fd);
}
}
void print_line_number(size_t line, const char sep) {
if (!opts.print_line_numbers) {
return;
}
if (opts.color) {
fprintf(out_fd, "%s%lu%s%c", opts.color_line_number, (unsigned long)line, color_reset, sep);
} else {
fprintf(out_fd, "%lu%c", (unsigned long)line, sep);
}
}
void print_column_number(const match_t matches[], size_t last_printed_match,
size_t prev_line_offset, const char sep) {
size_t column = 0;
if (prev_line_offset <= matches[last_printed_match].start) {
column = (matches[last_printed_match].start - prev_line_offset) + 1;
}
fprintf(out_fd, "%lu%c", (unsigned long)column, sep);
}
void print_file_separator(void) {
if (first_file_match == 0 && opts.print_break) {
fprintf(out_fd, "\n");
}
first_file_match = 0;
}
const char *normalize_path(const char *path) {
if (strlen(path) < 3) {
return path;
}
if (path[0] == '.' && path[1] == '/') {
return path + 2;
}
if (path[0] == '/' && path[1] == '/') {
return path + 1;
}
return path;
}
================================================
FILE: src/print.h
================================================
#ifndef PRINT_H
#define PRINT_H
#include "util.h"
void print_init_context(void);
void print_cleanup_context(void);
void print_context_append(const char *line, size_t len);
void print_trailing_context(const char *path, const char *buf, size_t n);
void print_path(const char *path, const char sep);
void print_path_count(const char *path, const char sep, const size_t count);
void print_line(const char *buf, size_t buf_pos, size_t prev_line_offset);
void print_binary_file_matches(const char *path);
void print_file_matches(const char *path, const char *buf, const size_t buf_len, const match_t matches[], const size_t matches_len);
void print_line_number(size_t line, const char sep);
void print_column_number(const match_t matches[], size_t last_printed_match,
size_t prev_line_offset, const char sep);
void print_file_separator(void);
const char *normalize_path(const char *path);
#ifdef _WIN32
void windows_use_ansi(int use_ansi);
int fprintf_w32(FILE *fp, const char *format, ...);
#endif
#endif
================================================
FILE: src/print_w32.c
================================================
#ifdef _WIN32
#include "print.h"
#include <io.h>
#include <stdarg.h>
#include <stdio.h>
#include <windows.h>
#ifndef FOREGROUND_MASK
#define FOREGROUND_MASK (FOREGROUND_RED | FOREGROUND_BLUE | FOREGROUND_GREEN | FOREGROUND_INTENSITY)
#endif
#ifndef BACKGROUND_MASK
#define BACKGROUND_MASK (BACKGROUND_RED | BACKGROUND_BLUE | BACKGROUND_GREEN | BACKGROUND_INTENSITY)
#endif
#define FG_RGB (FOREGROUND_RED | FOREGROUND_BLUE | FOREGROUND_GREEN)
#define BG_RGB (BACKGROUND_RED | BACKGROUND_BLUE | BACKGROUND_GREEN)
// BUFSIZ is guarenteed to be "at least 256 bytes" which might
// not be enough for us. Use an arbitrary but reasonably big
// buffer. win32 colored output will be truncated to this length.
#define BUF_SIZE (16 * 1024)
// max consecutive ansi sequence values beyond which we're aborting
// e.g. this is 3 values: \e[0;1;33m
#define MAX_VALUES 8
static int g_use_ansi = 0;
void windows_use_ansi(int use_ansi) {
g_use_ansi = use_ansi;
}
int fprintf_w32(FILE *fp, const char *format, ...) {
va_list args;
char buf[BUF_SIZE] = { 0 }, *ptr = buf;
static WORD attr_reset;
static BOOL attr_initialized = FALSE;
HANDLE stdo = INVALID_HANDLE_VALUE;
WORD attr;
DWORD written, csize;
CONSOLE_CURSOR_INFO cci;
CONSOLE_SCREEN_BUFFER_INFO csbi;
COORD coord;
// if we don't output to screen (tty) e.g. for pager/pipe or
// if for other reason we can't access the screen info, of if
// the user just prefers ansi, do plain passthrough.
BOOL passthrough =
g_use_ansi ||
!isatty(fileno(fp)) ||
INVALID_HANDLE_VALUE == (stdo = (HANDLE)_get_osfhandle(fileno(fp))) ||
!GetConsoleScreenBufferInfo(stdo, &csbi);
if (passthrough) {
int rv;
va_start(args, format);
rv = vfprintf(fp, format, args);
va_end(args);
return rv;
}
va_start(args, format);
// truncates to (null terminated) BUF_SIZE if too long.
// if too long - vsnprintf will fill count chars without
// terminating null. buf is zeroed, so make sure we don't fill it.
vsnprintf(buf, BUF_SIZE - 1, format, args);
va_end(args);
attr = csbi.wAttributes;
if (!attr_initialized) {
// reset is defined to have all (non color) attributes off
attr_reset = attr & (FG_RGB | BG_RGB);
attr_initialized = TRUE;
}
while (*ptr) {
if (*ptr == '\033') {
unsigned char c;
int i, n = 0, m = '\0', v[MAX_VALUES], w, h;
for (i = 0; i < MAX_VALUES; i++)
v[i] = -1;
ptr++;
retry:
if ((c = *ptr++) == 0)
break;
if (isdigit(c)) {
if (v[n] == -1)
v[n] = c - '0';
else
v[n] = v[n] * 10 + c - '0';
goto retry;
}
if (c == '[') {
goto retry;
}
if (c == ';') {
if (++n == MAX_VALUES)
break;
goto retry;
}
if (c == '>' || c == '?') {
m = c;
goto retry;
}
switch (c) {
// n is the last occupied index, so we have n+1 values
case 'h':
if (m == '?') {
for (i = 0; i <= n; i++) {
switch (v[i]) {
case 3:
GetConsoleScreenBufferInfo(stdo, &csbi);
w = csbi.dwSize.X;
h = csbi.srWindow.Bottom - csbi.srWindow.Top + 1;
csize = w * (h + 1);
coord.X = 0;
coord.Y = csbi.srWindow.Top;
FillConsoleOutputCharacter(stdo, ' ', csize, coord, &written);
FillConsoleOutputAttribute(stdo, csbi.wAttributes, csize, coord, &written);
SetConsoleCursorPosition(stdo, csbi.dwCursorPosition);
csbi.dwSize.X = 132;
SetConsoleScreenBufferSize(stdo, csbi.dwSize);
csbi.srWindow.Right = csbi.srWindow.Left + 131;
SetConsoleWindowInfo(stdo, TRUE, &csbi.srWindow);
break;
case 5:
attr =
((attr & FOREGROUND_MASK) << 4) |
((attr & BACKGROUND_MASK) >> 4);
SetConsoleTextAttribute(stdo, attr);
break;
case 9:
break;
case 25:
GetConsoleCursorInfo(stdo, &cci);
cci.bVisible = TRUE;
SetConsoleCursorInfo(stdo, &cci);
break;
case 47:
coord.X = 0;
coord.Y = 0;
SetConsoleCursorPosition(stdo, coord);
break;
default:
break;
}
}
} else if (m == '>' && v[0] == 5) {
GetConsoleCursorInfo(stdo, &cci);
cci.bVisible = FALSE;
SetConsoleCursorInfo(stdo, &cci);
}
break;
case 'l':
if (m == '?') {
for (i = 0; i <= n; i++) {
switch (v[i]) {
case 3:
GetConsoleScreenBufferInfo(stdo, &csbi);
w = csbi.dwSize.X;
h = csbi.srWindow.Bottom - csbi.srWindow.Top + 1;
csize = w * (h + 1);
coord.X = 0;
coord.Y = csbi.srWindow.Top;
FillConsoleOutputCharacter(stdo, ' ', csize, coord, &written);
FillConsoleOutputAttribute(stdo, csbi.wAttributes, csize, coord, &written);
SetConsoleCursorPosition(stdo, csbi.dwCursorPosition);
csbi.srWindow.Right = csbi.srWindow.Left + 79;
SetConsoleWindowInfo(stdo, TRUE, &csbi.srWindow);
csbi.dwSize.X = 80;
SetConsoleScreenBufferSize(stdo, csbi.dwSize);
break;
case 5:
attr =
((attr & FOREGROUND_MASK) << 4) |
((attr & BACKGROUND_MASK) >> 4);
SetConsoleTextAttribute(stdo, attr);
break;
case 25:
GetConsoleCursorInfo(stdo, &cci);
cci.bVisible = FALSE;
SetConsoleCursorInfo(stdo, &cci);
break;
default:
break;
}
}
} else if (m == '>' && v[0] == 5) {
GetConsoleCursorInfo(stdo, &cci);
cci.bVisible = TRUE;
SetConsoleCursorInfo(stdo, &cci);
}
break;
case 'm':
for (i = 0; i <= n; i++) {
if (v[i] == -1 || v[i] == 0)
attr = attr_reset;
else if (v[i] == 1)
attr |= FOREGROUND_INTENSITY;
else if (v[i] == 4)
attr |= FOREGROUND_INTENSITY;
else if (v[i] == 5) // blink is typically applied as bg intensity
attr |= BACKGROUND_INTENSITY;
else if (v[i] == 7)
attr =
((attr & FOREGROUND_MASK) << 4) |
((attr & BACKGROUND_MASK) >> 4);
else if (v[i] == 10)
; // symbol on
else if (v[i] == 11)
; // symbol off
else if (v[i] == 22)
attr &= ~FOREGROUND_INTENSITY;
else if (v[i] == 24)
attr &= ~FOREGROUND_INTENSITY;
else if (v[i] == 25)
attr &= ~BACKGROUND_INTENSITY;
else if (v[i] == 27)
attr =
((attr & FOREGROUND_MASK) << 4) |
((attr & BACKGROUND_MASK) >> 4);
else if (v[i] >= 30 && v[i] <= 37) {
attr &= ~FG_RGB; // doesn't affect attributes
if ((v[i] - 30) & 1)
attr |= FOREGROUND_RED;
if ((v[i] - 30) & 2)
attr |= FOREGROUND_GREEN;
if ((v[i] - 30) & 4)
attr |= FOREGROUND_BLUE;
} else if (v[i] == 39) // reset fg color and attributes
attr = (attr & ~FOREGROUND_MASK) | (attr_reset & FG_RGB);
else if (v[i] >= 40 && v[i] <= 47) {
attr &= ~BG_RGB;
if ((v[i] - 40) & 1)
attr |= BACKGROUND_RED;
if ((v[i] - 40) & 2)
attr |= BACKGROUND_GREEN;
if ((v[i] - 40) & 4)
attr |= BACKGROUND_BLUE;
} else if (v[i] == 49) // reset bg color
attr = (attr & ~BACKGROUND_MASK) | (attr_reset & BG_RGB);
}
SetConsoleTextAttribute(stdo, attr);
break;
case 'K':
GetConsoleScreenBufferInfo(stdo, &csbi);
coord = csbi.dwCursorPosition;
switch (v[0]) {
default:
case 0:
csize = csbi.dwSize.X - coord.X;
break;
case 1:
csize = coord.X;
coord.X = 0;
break;
case 2:
csize = csbi.dwSize.X;
coord.X = 0;
break;
}
FillConsoleOutputCharacter(stdo, ' ', csize, coord, &written);
FillConsoleOutputAttribute(stdo, csbi.wAttributes, csize, coord, &written);
SetConsoleCursorPosition(stdo, csbi.dwCursorPosition);
break;
case 'J':
GetConsoleScreenBufferInfo(stdo, &csbi);
w = csbi.dwSize.X;
h = csbi.srWindow.Bottom - csbi.srWindow.Top + 1;
coord = csbi.dwCursorPosition;
switch (v[0]) {
default:
case 0:
csize = w * (h - coord.Y) - coord.X;
coord.X = 0;
break;
case 1:
csize = w * coord.Y + coord.X;
coord.X = 0;
coord.Y = csbi.srWindow.Top;
break;
case 2:
csize = w * (h + 1);
coord.X = 0;
coord.Y = csbi.srWindow.Top;
break;
}
FillConsoleOutputCharacter(stdo, ' ', csize, coord, &written);
FillConsoleOutputAttribute(stdo, csbi.wAttributes, csize, coord, &written);
SetConsoleCursorPosition(stdo, csbi.dwCursorPosition);
break;
case 'H':
GetConsoleScreenBufferInfo(stdo, &csbi);
coord = csbi.dwCursorPosition;
if (v[0] != -1) {
if (v[1] != -1) {
coord.Y = csbi.srWindow.Top + v[0] - 1;
coord.X = v[1] - 1;
} else
coord.X = v[0] - 1;
} else {
coord.X = 0;
coord.Y = csbi.srWindow.Top;
}
if (coord.X < csbi.srWindow.Left)
coord.X = csbi.srWindow.Left;
else if (coord.X > csbi.srWindow.Right)
coord.X = csbi.srWindow.Right;
if (coord.Y < csbi.srWindow.Top)
coord.Y = csbi.srWindow.Top;
else if (coord.Y > csbi.srWindow.Bottom)
coord.Y = csbi.srWindow.Bottom;
SetConsoleCursorPosition(stdo, coord);
break;
default:
break;
}
} else {
putchar(*ptr);
ptr++;
}
}
return ptr - buf;
}
#endif /* _WIN32 */
================================================
FILE: src/scandir.c
================================================
#include <dirent.h>
#include <stdlib.h>
#include "scandir.h"
#include "util.h"
int ag_scandir(const char *dirname,
struct dirent ***namelist,
filter_fp filter,
void *baton) {
DIR *dirp = NULL;
struct dirent **names = NULL;
struct dirent *entry, *d;
int names_len = 32;
int results_len = 0;
dirp = opendir(dirname);
if (dirp == NULL) {
goto fail;
}
names = malloc(sizeof(struct dirent *) * names_len);
if (names == NULL) {
goto fail;
}
while ((entry = readdir(dirp)) != NULL) {
if ((*filter)(dirname, entry, baton) == FALSE) {
continue;
}
if (results_len >= names_len) {
struct dirent **tmp_names = names;
names_len *= 2;
names = realloc(names, sizeof(struct dirent *) * names_len);
if (names == NULL) {
free(tmp_names);
goto fail;
}
}
#if defined(__MINGW32__) || defined(__CYGWIN__)
d = malloc(sizeof(struct dirent));
#else
d = malloc(entry->d_reclen);
#endif
if (d == NULL) {
goto fail;
}
#if defined(__MINGW32__) || defined(__CYGWIN__)
memcpy(d, entry, sizeof(struct dirent));
#else
memcpy(d, entry, entry->d_reclen);
#endif
names[results_len] = d;
results_len++;
}
closedir(dirp);
*namelist = names;
return results_len;
fail:
if (dirp) {
closedir(dirp);
}
if (names != NULL) {
int i;
for (i = 0; i < results_len; i++) {
free(names[i]);
}
free(names);
}
return -1;
}
================================================
FILE: src/scandir.h
================================================
#ifndef SCANDIR_H
#define SCANDIR_H
#include "ignore.h"
typedef struct {
const ignores *ig;
const char *base_path;
size_t base_path_len;
const char *path_start;
} scandir_baton_t;
typedef int (*filter_fp)(const char *path, const struct dirent *, void *);
int ag_scandir(const char *dirname,
struct dirent ***namelist,
filter_fp filter,
void *baton);
#endif
================================================
FILE: src/search.c
================================================
#include "search.h"
#include "print.h"
#include "scandir.h"
size_t alpha_skip_lookup[256];
size_t *find_skip_lookup;
uint8_t h_table[H_SIZE] __attribute__((aligned(64)));
work_queue_t *work_queue = NULL;
work_queue_t *work_queue_tail = NULL;
int done_adding_files = 0;
pthread_cond_t files_ready = PTHREAD_COND_INITIALIZER;
pthread_mutex_t stats_mtx = PTHREAD_MUTEX_INITIALIZER;
pthread_mutex_t work_queue_mtx = PTHREAD_MUTEX_INITIALIZER;
symdir_t *symhash = NULL;
/* Returns: -1 if skipped, otherwise # of matches */
ssize_t search_buf(const char *buf, const size_t buf_len,
const char *dir_full_path) {
int binary = -1; /* 1 = yes, 0 = no, -1 = don't know */
size_t buf_offset = 0;
if (opts.search_stream) {
binary = 0;
} else if (!opts.search_binary_files && opts.mmap) { /* if not using mmap, binary files have already been skipped */
binary = is_binary((const void *)buf, buf_len);
if (binary) {
log_debug("File %s is binary. Skipping...", dir_full_path);
return -1;
}
}
size_t matches_len = 0;
match_t *matches;
size_t matches_size;
size_t matches_spare;
if (opts.invert_match) {
/* If we are going to invert the set of matches at the end, we will need
* one extra match struct, even if there are no matches at all. So make
* sure we have a nonempty array; and make sure we always have spare
* capacity for one extra.
*/
matches_size = 100;
matches = ag_malloc(matches_size * sizeof(match_t));
matches_spare = 1;
} else {
matches_size = 0;
matches = NULL;
matches_spare = 0;
}
if (!opts.literal && opts.query_len == 1 && opts.query[0] == '.') {
matches_size = 1;
matches = matches == NULL ? ag_malloc(matches_size * sizeof(match_t)) : matches;
matches[0].start = 0;
matches[0].end = buf_len;
matches_len = 1;
} else if (opts.literal) {
const char *match_ptr = buf;
while (buf_offset < buf_len) {
/* hash_strnstr only for little-endian platforms that allow unaligned access */
#if defined(__i386__) || defined(__x86_64__)
/* Decide whether to fall back on boyer-moore */
if ((size_t)opts.query_len < 2 * sizeof(uint16_t) - 1 || opts.query_len >= UCHAR_MAX) {
match_ptr = boyer_moore_strnstr(match_ptr, opts.query, buf_len - buf_offset, opts.query_len, alpha_skip_lookup, find_skip_lookup, opts.casing == CASE_INSENSITIVE);
} else {
match_ptr = hash_strnstr(match_ptr, opts.query, buf_len - buf_offset, opts.query_len, h_table, opts.casing == CASE_SENSITIVE);
}
#else
match_ptr = boyer_moore_strnstr(match_ptr, opts.query, buf_len - buf_offset, opts.query_len, alpha_skip_lookup, find_skip_lookup, opts.casing == CASE_INSENSITIVE);
#endif
if (match_ptr == NULL) {
break;
}
if (opts.word_regexp) {
const char *start = match_ptr;
const char *end = match_ptr + opts.query_len;
/* Check whether both start and end of the match lie on a word
* boundary
*/
if ((start == buf ||
is_wordchar(*(start - 1)) != opts.literal_starts_wordchar) &&
(end == buf + buf_len ||
is_wordchar(*end) != opts.literal_ends_wordchar)) {
/* It's a match */
} else {
/* It's not a match */
match_ptr += find_skip_lookup[0] - opts.query_len + 1;
buf_offset = match_ptr - buf;
continue;
}
}
realloc_matches(&matches, &matches_size, matches_len + matches_spare);
matches[matches_len].start = match_ptr - buf;
matches[matches_len].end = matches[matches_len].start + opts.query_len;
buf_offset = matches[matches_len].end;
log_debug("Match found. File %s, offset %lu bytes.", dir_full_path, matches[matches_len].start);
matches_len++;
match_ptr += opts.query_len;
if (opts.max_matches_per_file > 0 && matches_len >= opts.max_matches_per_file) {
log_err("Too many matches in %s. Skipping the rest of this file.", dir_full_path);
break;
}
}
} else {
int offset_vector[3];
if (opts.multiline) {
while (buf_offset < buf_len &&
(pcre_exec(opts.re, opts.re_extra, buf, buf_len, buf_offset, 0, offset_vector, 3)) >= 0) {
log_debug("Regex match found. File %s, offset %i bytes.", dir_full_path, offset_vector[0]);
buf_offset = offset_vector[1];
if (offset_vector[0] == offset_vector[1]) {
++buf_offset;
log_debug("Regex match is of length zero. Advancing offset one byte.");
}
realloc_matches(&matches, &matches_size, matches_len + matches_spare);
matches[matches_len].start = offset_vector[0];
matches[matches_len].end = offset_vector[1];
matches_len++;
if (opts.max_matches_per_file > 0 && matches_len >= opts.max_matches_per_file) {
log_err("Too many matches in %s. Skipping the rest of this file.", dir_full_path);
break;
}
}
} else {
while (buf_offset < buf_len) {
const char *line;
size_t line_len = buf_getline(&line, buf, buf_len, buf_offset);
if (!line) {
break;
}
size_t line_offset = 0;
while (line_offset < line_len) {
int rv = pcre_exec(opts.re, opts.re_extra, line, line_len, line_offset, 0, offset_vector, 3);
if (rv < 0) {
break;
}
size_t line_to_buf = buf_offset + line_offset;
log_debug("Regex match found. File %s, offset %i bytes.", dir_full_path, offset_vector[0]);
line_offset = offset_vector[1];
if (offset_vector[0] == offset_vector[1]) {
++line_offset;
log_debug("Regex match is of length zero. Advancing offset one byte.");
}
realloc_matches(&matches, &matches_size, matches_len + matches_spare);
matches[matches_len].start = offset_vector[0] + line_to_buf;
matches[matches_len].end = offset_vector[1] + line_to_buf;
matches_len++;
if (opts.max_matches_per_file > 0 && matches_len >= opts.max_matches_per_file) {
log_err("Too many matches in %s. Skipping the rest of this file.", dir_full_path);
goto multiline_done;
}
}
buf_offset += line_len + 1;
}
}
}
multiline_done:
if (opts.invert_match) {
matches_len = invert_matches(buf, buf_len, matches, matches_len);
}
if (opts.stats) {
pthread_mutex_lock(&stats_mtx);
stats.total_bytes += buf_len;
stats.total_files++;
stats.total_matches += matches_len;
if (matches_len > 0) {
stats.total_file_matches++;
}
pthread_mutex_unlock(&stats_mtx);
}
if (!opts.print_nonmatching_files && (matches_len > 0 || opts.print_all_paths)) {
if (binary == -1 && !opts.print_filename_only) {
binary = is_binary((const void *)buf, buf_len);
}
pthread_mutex_lock(&print_mtx);
if (opts.print_filename_only) {
if (opts.print_count) {
print_path_count(dir_full_path, opts.path_sep, (size_t)matches_len);
} else {
print_path(dir_full_path, opts.path_sep);
}
} else if (binary) {
print_binary_file_matches(dir_full_path);
} else {
print_file_matches(dir_full_path, buf, buf_len, matches, matches_len);
}
pthread_mutex_unlock(&print_mtx);
opts.match_found = 1;
} else if (opts.search_stream && opts.passthrough) {
fprintf(out_fd, "%s", buf);
} else {
log_debug("No match in %s", dir_full_path);
}
if (matches_len == 0 && opts.search_stream) {
print_context_append(buf, buf_len - 1);
}
if (matches_size > 0) {
free(matches);
}
/* FIXME: handle case where matches_len > SSIZE_MAX */
return (ssize_t)matches_len;
}
/* Return value: -1 if skipped, otherwise # of matches */
/* TODO: this will only match single lines. multi-line regexes silently don't match */
ssize_t search_stream(FILE *stream, const char *path) {
char *line = NULL;
ssize_t matches_count = 0;
ssize_t line_len = 0;
size_t line_cap = 0;
size_t i;
print_init_context();
for (i = 1; (line_len = getline(&line, &line_cap, stream)) > 0; i++) {
ssize_t result;
opts.stream_line_num = i;
result = search_buf(line, line_len, path);
if (result > 0) {
if (matches_count == -1) {
matches_count = 0;
}
matches_count += result;
} else if (matches_count <= 0 && result == -1) {
matches_count = -1;
}
if (line[line_len - 1] == '\n') {
line_len--;
}
print_trailing_context(path, line, line_len);
}
free(line);
print_cleanup_context();
return matches_count;
}
void search_file(const char *file_full_path) {
int fd = -1;
off_t f_len = 0;
char *buf = NULL;
struct stat statbuf;
int rv = 0;
int matches_count = -1;
FILE *fp = NULL;
rv = stat(file_full_path, &statbuf);
if (rv != 0) {
log_err("Skipping %s: Error fstat()ing file.", file_full_path);
goto cleanup;
}
if (opts.stdout_inode != 0 && opts.stdout_inode == statbuf.st_ino) {
log_debug("Skipping %s: stdout is redirected to it", file_full_path);
goto cleanup;
}
// handling only regular files and FIFOs
if (!S_ISREG(statbuf.st_mode) && !S_ISFIFO(statbuf.st_mode)) {
log_err("Skipping %s: Mode %u is not a file.", file_full_path, statbuf.st_mode);
goto cleanup;
}
fd = open(file_full_path, O_RDONLY);
if (fd < 0) {
/* XXXX: strerror is not thread-safe */
log_err("Skipping %s: Error opening file: %s", file_full_path, strerror(errno));
goto cleanup;
}
// repeating stat check with file handle to prevent TOCTOU issue
rv = fstat(fd, &statbuf);
if (rv != 0) {
log_err("Skipping %s: Error fstat()ing file.", file_full_path);
goto cleanup;
}
if (opts.stdout_inode != 0 && opts.stdout_inode == statbuf.st_ino) {
log_debug("Skipping %s: stdout is redirected to it", file_full_path);
goto cleanup;
}
// handling only regular files and FIFOs
if (!S_ISREG(statbuf.st_mode) && !S_ISFIFO(statbuf.st_mode)) {
log_err("Skipping %s: Mode %u is not a file.", file_full_path, statbuf.st_mode);
goto cleanup;
}
print_init_context();
if (statbuf.st_mode & S_IFIFO) {
log_debug("%s is a named pipe. stream searching", file_full_path);
fp = fdopen(fd, "r");
matches_count = search_stream(fp, file_full_path);
fclose(fp);
goto cleanup;
}
f_len = statbuf.st_size;
if (f_len == 0) {
if (opts.query[0] == '.' && opts.query_len == 1 && !opts.literal && opts.search_all_files) {
matches_count = search_buf(buf, f_len, file_full_path);
} else {
log_debug("Skipping %s: file is empty.", file_full_path);
}
goto cleanup;
}
if (!opts.literal && f_len > INT_MAX) {
log_err("Skipping %s: pcre_exec() can't handle files larger than %i bytes.", file_full_path, INT_MAX);
goto cleanup;
}
#ifdef _WIN32
{
HANDLE hmmap = CreateFileMapping(
(HANDLE)_get_osfhandle(fd), 0, PAGE_READONLY, 0, f_len, NULL);
buf = (char *)MapViewOfFile(hmmap, FILE_SHARE_READ, 0, 0, f_len);
if (hmmap != NULL)
CloseHandle(hmmap);
}
if (buf == NULL) {
FormatMessageA(
FORMAT_MESSAGE_ALLOCATE_BUFFER |
FORMAT_MESSAGE_FROM_SYSTEM |
FORMAT_MESSAGE_IGNORE_INSERTS,
NULL, GetLastError(), 0, (void *)&buf, 0, NULL);
log_err("File %s failed to load: %s.", file_full_path, buf);
LocalFree((void *)buf);
goto cleanup;
}
#else
if (opts.mmap) {
buf = mmap(0, f_len, PROT_READ, MAP_PRIVATE, fd, 0);
if (buf == MAP_FAILED) {
log_err("File %s failed to load: %s.", file_full_path, strerror(errno));
goto cleanup;
}
#if HAVE_MADVISE
madvise(buf, f_len, MADV_SEQUENTIAL);
#elif HAVE_POSIX_FADVISE
posix_fadvise(fd, 0, f_len, POSIX_MADV_SEQUENTIAL);
#endif
} else {
buf = ag_malloc(f_len);
ssize_t bytes_read = 0;
if (!opts.search_binary_files) {
bytes_read += read(fd, buf, ag_min(f_len, 512));
// Optimization: If skipping binary files, don't read the whole buffer before checking if binary or not.
if (is_binary(buf, f_len)) {
log_debug("File %s is binary. Skipping...", file_full_path);
goto cleanup;
}
}
while (bytes_read < f_len) {
bytes_read += read(fd, buf + bytes_read, f_len);
}
if (bytes_read != f_len) {
die("File %s read(): expected to read %u bytes but read %u", file_full_path, f_len, bytes_read);
}
}
#endif
if (opts.search_zip_files) {
ag_compression_type zip_type = is_zipped(buf, f_len);
if (zip_type != AG_NO_COMPRESSION) {
#if HAVE_FOPENCOOKIE
log_debug("%s is a compressed file. stream searching", file_full_path);
fp = decompress_open(fd, "r", zip_type);
matches_count = search_stream(fp, file_full_path);
fclose(fp);
#else
int _buf_len = (int)f_len;
char *_buf = decompress(zip_type, buf, f_len, file_full_path, &_buf_len);
if (_buf == NULL || _buf_len == 0) {
log_err("Cannot decompress zipped file %s", file_full_path);
goto cleanup;
}
matches_count = search_buf(_buf, _buf_len, file_full_path);
free(_buf);
#endif
goto cleanup;
}
}
matches_count = search_buf(buf, f_len, file_full_path);
cleanup:
if (opts.print_nonmatching_files && matches_count == 0) {
pthread_mutex_lock(&print_mtx);
print_path(file_full_path, opts.path_sep);
pthread_mutex_unlock(&print_mtx);
opts.match_found = 1;
}
print_cleanup_context();
if (buf != NULL) {
#ifdef _WIN32
UnmapViewOfFile(buf);
#else
if (opts.mmap) {
if (buf != MAP_FAILED) {
munmap(buf, f_len);
}
} else {
free(buf);
}
#endif
}
if (fd != -1) {
close(fd);
}
}
void *search_file_worker(void *i) {
work_queue_t *queue_item;
int worker_id = *(int *)i;
log_debug("Worker %i started", worker_id);
while (TRUE) {
pthread_mutex_lock(&work_queue_mtx);
while (work_queue == NULL) {
if (done_adding_files) {
pthread_mutex_unlock(&work_queue_mtx);
log_debug("Worker %i finished.", worker_id);
pthread_exit(NULL);
}
pthread_cond_wait(&files_ready, &work_queue_mtx);
}
queue_item = work_queue;
work_queue = work_queue->next;
if (work_queue == NULL) {
work_queue_tail = NULL;
}
pthread_mutex_unlock(&work_queue_mtx);
search_file(queue_item->path);
free(queue_item->path);
free(queue_item);
}
}
static int check_symloop_enter(const char *path, dirkey_t *outkey) {
#ifdef _WIN32
gitextract_qvo96f4r/ ├── .clang-format ├── .gitignore ├── .travis.yml ├── CONTRIBUTING.md ├── LICENSE ├── Makefile.am ├── Makefile.w32 ├── NOTICE ├── README.md ├── _the_silver_searcher ├── ag.bashcomp.sh ├── autogen.sh ├── build.sh ├── configure.ac ├── doc/ │ ├── ag.1 │ ├── ag.1.md │ └── generate_man.sh ├── format.sh ├── m4/ │ └── ax_pthread.m4 ├── pgo.sh ├── sanitize.sh ├── src/ │ ├── decompress.c │ ├── decompress.h │ ├── ignore.c │ ├── ignore.h │ ├── lang.c │ ├── lang.h │ ├── log.c │ ├── log.h │ ├── main.c │ ├── options.c │ ├── options.h │ ├── print.c │ ├── print.h │ ├── print_w32.c │ ├── scandir.c │ ├── scandir.h │ ├── search.c │ ├── search.h │ ├── uthash.h │ ├── util.c │ ├── util.h │ ├── win32/ │ │ └── config.h │ └── zfile.c ├── tests/ │ ├── adjacent_matches.t │ ├── bad_path.t │ ├── big/ │ │ ├── big_file.t │ │ └── create_big_file.py │ ├── case_sensitivity.t │ ├── color.t │ ├── column.t │ ├── count.t │ ├── ds_store_ignore.t │ ├── empty_environment.t │ ├── empty_match.t │ ├── exitcodes.t │ ├── fail/ │ │ ├── unicode_case_insensitive.t │ │ └── unicode_case_insensitive.t.err │ ├── files_with_matches.t │ ├── filetype.t │ ├── hidden_option.t │ ├── ignore_abs_path.t │ ├── ignore_absolute_search_path_with_glob.t │ ├── ignore_backups.t │ ├── ignore_examine_parent_ignorefiles.t │ ├── ignore_extensions.t │ ├── ignore_gitignore.t │ ├── ignore_invert.t │ ├── ignore_pattern_in_subdirectory.t │ ├── ignore_slash_in_subdir.t │ ├── ignore_subdir.t │ ├── ignore_vcs.t │ ├── invert_match.t │ ├── is_binary_pdf.t │ ├── line_width.t │ ├── list_file_types.t │ ├── literal_word_regexp.t │ ├── max_count.t │ ├── multiline.t │ ├── negated_options.t │ ├── one_device.t │ ├── only_matching.t │ ├── option_g.t │ ├── option_smartcase.t │ ├── passthrough.t │ ├── pipecontext.t │ ├── print_all_files.t │ ├── print_end.t │ ├── print_end.txt │ ├── search_stdin.t │ ├── setup.sh │ ├── stupid_fnmatch.t.disabled │ ├── vimgrep.t │ └── word_regexp.t └── the_silver_searcher.spec.in
SYMBOL INDEX (112 symbols across 22 files)
FILE: src/decompress.c
function ag_compression_type (line 220) | ag_compression_type is_zipped(const void *buf, const int buf_len) {
FILE: src/decompress.h
type ag_compression_type (line 10) | typedef enum {
FILE: src/ignore.c
function is_empty (line 42) | int is_empty(ignores *ig) {
function ignores (line 46) | ignores *init_ignore(ignores *parent, const char *dirname, const size_t ...
function cleanup_ignore (line 83) | void cleanup_ignore(ignores *ig) {
function add_ignore_pattern (line 99) | void add_ignore_pattern(ignores *ig, const char *pattern) {
function load_ignore_patterns (line 173) | void load_ignore_patterns(ignores *ig, const char *path) {
function ackmate_dir_match (line 200) | static int ackmate_dir_match(const char *dir_name) {
function path_ignore_search (line 209) | static int path_ignore_search(const ignores *ig, const char *path, const...
function filename_filter (line 292) | int filename_filter(const char *path, const struct dirent *dir, void *ba...
FILE: src/ignore.h
type ignores (line 7) | struct ignores {
type ignores (line 30) | typedef struct ignores ignores;
type dirent (line 44) | struct dirent
FILE: src/lang.c
function get_lang_count (line 148) | size_t get_lang_count() {
function combine_file_extensions (line 184) | size_t combine_file_extensions(size_t *extension_index, size_t len, char...
FILE: src/lang.h
type lang_spec_t (line 7) | typedef struct {
FILE: src/log.c
type log_level (line 8) | enum log_level
function set_log_level (line 10) | void set_log_level(enum log_level threshold) {
function log_debug (line 14) | void log_debug(const char *fmt, ...) {
function log_msg (line 21) | void log_msg(const char *fmt, ...) {
function log_warn (line 28) | void log_warn(const char *fmt, ...) {
function log_err (line 35) | void log_err(const char *fmt, ...) {
function vplog (line 42) | void vplog(const unsigned int level, const char *fmt, va_list args) {
function plog (line 71) | void plog(const unsigned int level, const char *fmt, ...) {
FILE: src/log.h
type log_level (line 14) | enum log_level {
type log_level (line 22) | enum log_level
FILE: src/main.c
type worker_t (line 31) | typedef struct {
function main (line 36) | int main(int argc, char **argv) {
FILE: src/options.c
function usage (line 26) | void usage(void) {
function print_version (line 127) | void print_version(void) {
function init_options (line 147) | void init_options(void) {
function cleanup_options (line 179) | void cleanup_options(void) {
function parse_options (line 209) | void parse_options(int argc, char **argv, char **base_paths[], char **pa...
FILE: src/options.h
type case_behavior (line 13) | enum case_behavior {
type path_print_behavior (line 21) | enum path_print_behavior {
type cli_options (line 29) | typedef struct {
type option_t (line 97) | typedef struct option option_t;
FILE: src/print.c
type print_context (line 23) | struct print_context {
function print_init_context (line 36) | void print_init_context(void) {
function print_cleanup_context (line 52) | void print_cleanup_context(void) {
function print_context_append (line 68) | void print_context_append(const char *line, size_t len) {
function print_trailing_context (line 79) | void print_trailing_context(const char *path, const char *buf, size_t n) {
function print_path (line 103) | void print_path(const char *path, const char sep) {
function print_path_count (line 122) | void print_path_count(const char *path, const char sep, const size_t cou...
function print_line (line 133) | void print_line(const char *buf, size_t buf_pos, size_t prev_line_offset) {
function print_binary_file_matches (line 142) | void print_binary_file_matches(const char *path) {
function print_file_matches (line 148) | void print_file_matches(const char *path, const char *buf, const size_t ...
function print_line_number (line 334) | void print_line_number(size_t line, const char sep) {
function print_column_number (line 345) | void print_column_number(const match_t matches[], size_t last_printed_ma...
function print_file_separator (line 354) | void print_file_separator(void) {
FILE: src/print_w32.c
function windows_use_ansi (line 29) | void windows_use_ansi(int use_ansi) {
function fprintf_w32 (line 33) | int fprintf_w32(FILE *fp, const char *format, ...) {
FILE: src/scandir.c
function ag_scandir (line 7) | int ag_scandir(const char *dirname,
FILE: src/scandir.h
type scandir_baton_t (line 6) | typedef struct {
type dirent (line 13) | struct dirent
type dirent (line 16) | struct dirent
FILE: src/search.c
function search_buf (line 19) | ssize_t search_buf(const char *buf, const size_t buf_len,
function search_stream (line 229) | ssize_t search_stream(FILE *stream, const char *path) {
function search_file (line 261) | void search_file(const char *file_full_path) {
function check_symloop_enter (line 472) | static int check_symloop_enter(const char *path, dirkey_t *outkey) {
function check_symloop_leave (line 505) | static int check_symloop_leave(dirkey_t *dirkey) {
function search_dir (line 530) | void search_dir(ignores *ig, const char *base_path, const char *path, co...
FILE: src/search.h
type work_queue_t (line 38) | struct work_queue_t {
type work_queue_t (line 42) | typedef struct work_queue_t work_queue_t;
type dirkey_t (line 57) | typedef struct {
type symdir_t (line 62) | typedef struct {
FILE: src/uthash.h
type UT_hash_bucket (line 903) | typedef struct UT_hash_bucket {
type UT_hash_table (line 927) | typedef struct UT_hash_table {
type UT_hash_handle (line 960) | typedef struct UT_hash_handle {
FILE: src/util.c
function free_strings (line 58) | void free_strings(char **strs, const size_t strs_len) {
function generate_alpha_skip (line 69) | void generate_alpha_skip(const char *find, size_t f_len, size_t skip_loo...
function is_prefix (line 88) | int is_prefix(const char *s, const size_t s_len, const size_t pos, const...
function suffix_len (line 106) | size_t suffix_len(const char *s, const size_t s_len, const size_t pos, c...
function generate_find_skip (line 124) | void generate_find_skip(const char *find, const size_t f_len, size_t **s...
function ag_max (line 146) | size_t ag_max(size_t a, size_t b) {
function ag_min (line 153) | size_t ag_min(size_t a, size_t b) {
function generate_hash (line 160) | void generate_hash(const char *find, const size_t f_len, uint8_t *h_tabl...
function NO_SANITIZE_ALIGNMENT (line 206) | NO_SANITIZE_ALIGNMENT const char *hash_strnstr(const char *s, const char...
function invert_matches (line 242) | size_t invert_matches(const char *buf, const size_t buf_len, match_t mat...
function realloc_matches (line 307) | void realloc_matches(match_t **matches, size_t *matches_size, size_t mat...
function compile_study (line 316) | void compile_study(pcre **re, pcre_extra **re_extra, char *q, const int ...
function is_binary (line 333) | int is_binary(const void *buf, const size_t buf_len) {
function is_regex (line 388) | int is_regex(const char *query) {
function is_fnmatch (line 408) | int is_fnmatch(const char *filename) {
function binary_search (line 421) | int binary_search(const char *needle, char **haystack, int start, int en...
function init_wordchar_table (line 443) | void init_wordchar_table(void) {
function is_wordchar (line 455) | int is_wordchar(char ch) {
function is_lowercase (line 459) | int is_lowercase(const char *s) {
function is_directory (line 469) | int is_directory(const char *path, const struct dirent *d) {
function is_symlink (line 494) | int is_symlink(const char *path, const struct dirent *d) {
function is_named_pipe (line 519) | int is_named_pipe(const char *path, const struct dirent *d) {
function ag_asprintf (line 540) | void ag_asprintf(char **ret, const char *fmt, ...) {
function die (line 549) | void die(const char *fmt, ...) {
function getline (line 592) | ssize_t getline(char **lineptr, size_t *n, FILE *stream) {
function buf_getline (line 626) | ssize_t buf_getline(const char **line, const char *buf, const size_t buf...
function strlcpy (line 655) | size_t strlcpy(char *dst, const char *src, size_t size) {
function vasprintf (line 684) | int vasprintf(char **ret, const char *fmt, va_list args) {
FILE: src/util.h
type match_t (line 39) | typedef struct {
type ag_stats (line 44) | typedef struct {
type word_t (line 57) | typedef union {
type dirent (line 93) | struct dirent
type dirent (line 94) | struct dirent
type dirent (line 95) | struct dirent
FILE: src/zfile.c
type _off64_t (line 7) | typedef _off64_t off64_t;
type zfile (line 53) | struct zfile {
function zfile_cookie_init (line 75) | static int
function zfile_cookie_cleanup (line 127) | static void
function FILE (line 150) | FILE *
function zfile_read (line 205) | static ssize_t
function zfile_seek (line 333) | static int
function zfile_close (line 392) | static int
FILE: tests/big/create_big_file.py
function create_big_file (line 15) | def create_big_file():
Condensed preview — 95 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (365K chars).
[
{
"path": ".clang-format",
"chars": 1178,
"preview": "AlignEscapedNewlinesLeft: false\nAlignTrailingComments: true\nAllowAllParametersOfDeclarationOnNextLine: true\nAllowShortBl"
},
{
"path": ".gitignore",
"chars": 359,
"preview": "*.dSYM\n*.gcda\n*.o\n*.plist\n.deps\n.dirstamp\n.DS_Store\naclocal.m4\nag\nag.exe\nautom4te.cache\ncachegrind.out.*\ncallgrind.out.*"
},
{
"path": ".travis.yml",
"chars": 886,
"preview": "language: c\ndist: xenial\nsudo: false\n\nbranches:\n only:\n - master\n - ppc64le\narch:\n - amd64\n - ppc64le\n\ncompiler"
},
{
"path": "CONTRIBUTING.md",
"chars": 1503,
"preview": "## Contributing\n\nI like when people send pull requests. It validates my existence. If you want to help out, check the [i"
},
{
"path": "LICENSE",
"chars": 11358,
"preview": "\n Apache License\n Version 2.0, January 2004\n "
},
{
"path": "Makefile.am",
"chars": 995,
"preview": "ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS}\n\nbin_PROGRAMS = ag\nag_SOURCES = src/ignore.c src/ignore.h src/log.c src/log.h src/opt"
},
{
"path": "Makefile.w32",
"chars": 668,
"preview": "SED=sed\nVERSION:=$(shell \"$(SED)\" -n \"s/[^[]*\\[\\([0-9]\\+\\.[0-9]\\+\\.[0-9]\\+\\)\\],/\\1/p\" configure.ac)\n\nCC=gcc\nRM=/bin/rm\n\n"
},
{
"path": "NOTICE",
"chars": 52,
"preview": "The Silver Searcher\nCopyright 2011-2016 Geoff Greer\n"
},
{
"path": "README.md",
"chars": 7985,
"preview": "# The Silver Searcher\n\nA code searching tool similar to `ack`, with a focus on speed.\n\n[![Build Status](https://travis-c"
},
{
"path": "_the_silver_searcher",
"chars": 4980,
"preview": "#compdef ag\n\n# Completion function for zsh\n\nlocal ret=1\nlocal -a args expl\n\n# Intentionally avoided many possible mutual"
},
{
"path": "ag.bashcomp.sh",
"chars": 2471,
"preview": "_ag() {\n local lngopt shtopt split=false\n local cur prev\n\n COMPREPLY=()\n cur=$(_get_cword \"=\")\n prev=\"${COMP_WORDS["
},
{
"path": "autogen.sh",
"chars": 475,
"preview": "#!/bin/sh\n\nset -e\ncd \"$(dirname \"$0\")\"\n\nAC_SEARCH_OPTS=\"\"\n# For those of us with pkg-config and other tools in /usr/loca"
},
{
"path": "build.sh",
"chars": 79,
"preview": "#!/bin/sh\n\nset -e\ncd \"$(dirname \"$0\")\"\n\n./autogen.sh\n./configure \"$@\"\nmake -j4\n"
},
{
"path": "configure.ac",
"chars": 2740,
"preview": "AC_INIT(\n [the_silver_searcher],\n [2.2.0],\n [https://github.com/ggreer/the_silver_searcher/issues],\n [the_si"
},
{
"path": "doc/ag.1",
"chars": 8407,
"preview": ".\\\" generated with Ronn/v0.7.3\n.\\\" http://github.com/rtomayko/ronn/tree/0.7.3\n.\n.TH \"AG\" \"1\" \"December 2016\" \"\" \"\"\n.\n.SH"
},
{
"path": "doc/ag.1.md",
"chars": 7914,
"preview": "ag(1) -- The Silver Searcher. Like ack, but faster.\n=============================================\n\n## SYNOPSIS\n\n`ag` [_o"
},
{
"path": "doc/generate_man.sh",
"chars": 269,
"preview": "#!/bin/sh\n\n# ronn is used to turn the markdown into a manpage.\n# Get ronn at https://github.com/rtomayko/ronn\n# Alternat"
},
{
"path": "format.sh",
"chars": 1291,
"preview": "#!/bin/bash\n\nfunction usage() {\n echo \"Usage: $0 test|reformat\"\n}\n\nif [ $# -eq 0 ]\nthen\n usage\n exit 0\nfi\n\nif ["
},
{
"path": "m4/ax_pthread.m4",
"chars": 13758,
"preview": "# ===========================================================================\n# http://www.gnu.org/software/autoc"
},
{
"path": "pgo.sh",
"chars": 188,
"preview": "#!/bin/sh\n\nset -e\ncd \"$(dirname \"$0\")\"\n\nmake clean\n./build.sh CFLAGS=\"$CFLAGS -fprofile-generate\"\n./ag example ..\nmake c"
},
{
"path": "sanitize.sh",
"chars": 4940,
"preview": "#!/bin/bash\n# Copyright 2016 Allen Wild\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may no"
},
{
"path": "src/decompress.c",
"chars": 8330,
"preview": "#include <string.h>\n#include <unistd.h>\n\n#include \"decompress.h\"\n\n#ifdef HAVE_LZMA_H\n#include <lzma.h>\n\n/* http://tukaa"
},
{
"path": "src/decompress.h",
"chars": 553,
"preview": "#ifndef DECOMPRESS_H\n#define DECOMPRESS_H\n\n#include <stdio.h>\n\n#include \"config.h\"\n#include \"log.h\"\n#include \"options.h\""
},
{
"path": "src/ignore.c",
"chars": 11889,
"preview": "#include <ctype.h>\n#include <dirent.h>\n#include <limits.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#i"
},
{
"path": "src/ignore.h",
"chars": 1231,
"preview": "#ifndef IGNORE_H\n#define IGNORE_H\n\n#include <dirent.h>\n#include <sys/types.h>\n\nstruct ignores {\n char **extensions; /"
},
{
"path": "src/lang.c",
"chars": 7419,
"preview": "#include <stdlib.h>\n#include <string.h>\n\n#include \"lang.h\"\n#include \"util.h\"\n\nlang_spec_t langs[] = {\n { \"actionscrip"
},
{
"path": "src/lang.h",
"chars": 860,
"preview": "#ifndef LANG_H\n#define LANG_H\n\n#define MAX_EXTENSIONS 12\n#define SINGLE_EXT_LEN 20\n\ntypedef struct {\n const char *nam"
},
{
"path": "src/log.c",
"chars": 1670,
"preview": "#include <stdarg.h>\n#include <stdio.h>\n\n#include \"log.h\"\n#include \"util.h\"\n\npthread_mutex_t print_mtx = PTHREAD_MUTEX_IN"
},
{
"path": "src/log.h",
"chars": 633,
"preview": "#ifndef LOG_H\n#define LOG_H\n\n#include <stdarg.h>\n\n#include \"config.h\"\n\n#ifdef HAVE_PTHREAD_H\n#include <pthread.h>\n#endif"
},
{
"path": "src/main.c",
"chars": 7392,
"preview": "#include <ctype.h>\n#include <pcre.h>\n#include <stdarg.h>\n#include <stdio.h>\n#include <string.h>\n#include <sys/time.h>\n#i"
},
{
"path": "src/options.c",
"chars": 32019,
"preview": "#include <errno.h>\n#include <limits.h>\n#include <stdarg.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#i"
},
{
"path": "src/options.h",
"chars": 2719,
"preview": "#ifndef OPTIONS_H\n#define OPTIONS_H\n\n#include <getopt.h>\n#include <sys/stat.h>\n\n#include <pcre.h>\n\n#define DEFAULT_AFTER"
},
{
"path": "src/print.c",
"chars": 14334,
"preview": "#include <limits.h>\n#include <stdarg.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n\n#include \"ignore.h\"\n"
},
{
"path": "src/print.h",
"chars": 1029,
"preview": "#ifndef PRINT_H\n#define PRINT_H\n\n#include \"util.h\"\n\nvoid print_init_context(void);\nvoid print_cleanup_context(void);\nvoi"
},
{
"path": "src/print_w32.c",
"chars": 14355,
"preview": "#ifdef _WIN32\n\n#include \"print.h\"\n#include <io.h>\n#include <stdarg.h>\n#include <stdio.h>\n#include <windows.h>\n\n#ifndef F"
},
{
"path": "src/scandir.c",
"chars": 1689,
"preview": "#include <dirent.h>\n#include <stdlib.h>\n\n#include \"scandir.h\"\n#include \"util.h\"\n\nint ag_scandir(const char *dirname,\n "
},
{
"path": "src/scandir.h",
"chars": 423,
"preview": "#ifndef SCANDIR_H\n#define SCANDIR_H\n\n#include \"ignore.h\"\n\ntypedef struct {\n const ignores *ig;\n const char *base_p"
},
{
"path": "src/search.c",
"chars": 24343,
"preview": "#include \"search.h\"\n#include \"print.h\"\n#include \"scandir.h\"\n\nsize_t alpha_skip_lookup[256];\nsize_t *find_skip_lookup;\nui"
},
{
"path": "src/search.h",
"chars": 1600,
"preview": "#ifndef SEARCH_H\n#define SEARCH_H\n\n#include <dirent.h>\n#include <errno.h>\n#include <fcntl.h>\n#include <limits.h>\n#includ"
},
{
"path": "src/uthash.h",
"chars": 75812,
"preview": "/*\nCopyright (c) 2003-2014, Troy D. Hanson http://troydhanson.github.com/uthash/\nAll rights reserved.\n\nRedistributio"
},
{
"path": "src/util.c",
"chars": 18961,
"preview": "#include <ctype.h>\n#include <stdarg.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#include <sys/stat.h>\n"
},
{
"path": "src/util.h",
"chars": 3634,
"preview": "#ifndef UTIL_H\n#define UTIL_H\n\n#include <dirent.h>\n#include <pcre.h>\n#include <stdint.h>\n#include <stdio.h>\n#include <st"
},
{
"path": "src/win32/config.h",
"chars": 43,
"preview": "#define HAVE_LZMA_H\n#define HAVE_PTHREAD_H\n"
},
{
"path": "src/zfile.c",
"chars": 10501,
"preview": "#ifdef __FreeBSD__\n#include <sys/endian.h>\n#endif\n#include <sys/types.h>\n\n#ifdef __CYGWIN__\ntypedef _off64_t off64_t;\n#e"
},
{
"path": "tests/adjacent_matches.t",
"chars": 309,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ alias ag=\"$TESTDIR/../ag --noaffinity --workers=1 --parallel --color\"\n $ printf 'bl"
},
{
"path": "tests/bad_path.t",
"chars": 205,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n\nComplain about nonexistent path:\n\n $ ag foo doesnt_exist\n ERR: Error stat()ing: doesn"
},
{
"path": "tests/big/big_file.t",
"chars": 667,
"preview": "Setup and create really big file:\n\n $ . $TESTDIR/../setup.sh\n $ python3 $TESTDIR/create_big_file.py $TESTDIR/big_file."
},
{
"path": "tests/big/create_big_file.py",
"chars": 638,
"preview": "#!/usr/bin/env python\n\n# Create an 8GB file of mostly \"abcdefghijklmnopqrstuvwxyz01234\",\n# with a few instances of \"hell"
},
{
"path": "tests/case_sensitivity.t",
"chars": 604,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'Foo\\n' >> ./sample\n $ printf 'bar\\n' >> ./sample\n\nSmart case by default:\n\n "
},
{
"path": "tests/color.t",
"chars": 606,
"preview": "Setup. Note that we have to turn --color on manually since ag detects that\nstdout isn't a tty when running in cram.\n\n $"
},
{
"path": "tests/column.t",
"chars": 342,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf \"blah\\nblah2\\n\" > blah.txt\n\nEnsure column is correct:\n\n $ ag --column \"blah\\"
},
{
"path": "tests/count.t",
"chars": 549,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ unalias ag\n $ alias ag=\"$TESTDIR/../ag --noaffinity --nocolor --workers=1\"\n $ prin"
},
{
"path": "tests/ds_store_ignore.t",
"chars": 245,
"preview": "Setup.\n $ . $TESTDIR/setup.sh\n $ mkdir -p dir0/dir1/dir2\n $ printf '*.DS_Store\\n' > dir0/.ignore\n $ printf 'blah\\n' "
},
{
"path": "tests/empty_environment.t",
"chars": 216,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf \"hello world\\n\" >test.txt\n\nVerify ag runs with an empty environment:\n\n $ env"
},
{
"path": "tests/empty_match.t",
"chars": 539,
"preview": "Setup.\n $ . $TESTDIR/setup.sh\n $ touch empty.txt\n $ printf 'foo\\n' > nonempty.txt\n\nZero-length match on an empty file"
},
{
"path": "tests/exitcodes.t",
"chars": 366,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'foo\\n' > ./exitcodes_test.txt\n $ printf 'bar\\n' >> ./exitcodes_test.txt\n\nNo"
},
{
"path": "tests/fail/unicode_case_insensitive.t",
"chars": 357,
"preview": "Setup:\n\n $ . $TESTDIR/../setup.sh\n $ printf \"hello=你好\\n\" > test.txt\n $ printf \"hello=你好\\n\" >> test.txt\n\nNormal search"
},
{
"path": "tests/fail/unicode_case_insensitive.t.err",
"chars": 469,
"preview": "Setup:\n\n $ . $TESTDIR/../setup.sh\n $ printf \"hello=你好\\n\" > test.txt\n $ printf \"hello=你好\\n\" >> test.txt\n\nNormal search"
},
{
"path": "tests/files_with_matches.t",
"chars": 1717,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'foo\\n' > ./foo.txt\n $ printf 'bar\\n' > ./bar.txt\n $ printf 'foo\\nbar\\nbaz\\"
},
{
"path": "tests/filetype.t",
"chars": 719,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ TEST_FILETYPE_EXT1=`ag --list-file-types | grep -E '^[ \\t]+\\..+' | head -n 1 | awk '"
},
{
"path": "tests/hidden_option.t",
"chars": 639,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir hidden_bug\n $ cd hidden_bug\n $ printf \"test\\n\" > a.txt\n $ git init --quiet\n"
},
{
"path": "tests/ignore_abs_path.t",
"chars": 505,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir -p ./a/b/c\n $ printf 'whatever1\\n' > ./a/b/c/blah.yml\n $ printf 'whatever2\\n"
},
{
"path": "tests/ignore_absolute_search_path_with_glob.t",
"chars": 323,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir -p parent/multi-part\n $ printf 'match1\\n' > parent/multi-part/file1.txt\n $ p"
},
{
"path": "tests/ignore_backups.t",
"chars": 1515,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir -p ./a/b/c\n $ printf 'whatever1\\n' > ./a/b/c/foo.yml\n $ printf 'whatever2\\n"
},
{
"path": "tests/ignore_examine_parent_ignorefiles.t",
"chars": 433,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir -p subdir\n $ printf 'match1\\n' > subdir/file1.txt\n $ printf 'file1.txt\\n' > "
},
{
"path": "tests/ignore_extensions.t",
"chars": 1162,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf '*.js\\n' > .ignore\n $ printf '*.test.txt\\n' >> .ignore\n $ printf 'targetA\\n"
},
{
"path": "tests/ignore_gitignore.t",
"chars": 326,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ export HOME=$PWD\n $ printf '[core]\\nexcludesfile = ~/.gitignore.global' >> $HOME/.g"
},
{
"path": "tests/ignore_invert.t",
"chars": 261,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'blah1\\n' > ./printme.txt\n $ printf 'blah2\\n' > ./dontprintme.c\n $ printf '"
},
{
"path": "tests/ignore_pattern_in_subdirectory.t",
"chars": 299,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir subdir\n $ printf 'first\\n' > file1.txt\n $ printf 'second\\n' > subdir/file2.t"
},
{
"path": "tests/ignore_slash_in_subdir.t",
"chars": 402,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir -p subdir/ignoredir\n $ mkdir ignoredir\n $ printf 'match1\\n' > subdir/ignored"
},
{
"path": "tests/ignore_subdir.t",
"chars": 475,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir -p ./a/b/c\n $ printf 'whatever1\\n' > ./a/b/c/blah.yml\n $ printf 'whatever2\\n"
},
{
"path": "tests/ignore_vcs.t",
"chars": 432,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'whatever1\\n' > ./always.txt\n $ printf 'whatever2\\n' > ./git.txt\n $ printf "
},
{
"path": "tests/invert_match.t",
"chars": 444,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'valid: 1\\n' > ./blah.txt\n $ printf 'some_string\\n' >> ./blah.txt\n $ printf"
},
{
"path": "tests/is_binary_pdf.t",
"chars": 226,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ cp $TESTDIR/is_binary.pdf .\n\nPDF files are binary. Do not search them by default:\n\n "
},
{
"path": "tests/line_width.t",
"chars": 785,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf \"12345678901234567890123456789012345678901234567890\\n\" >> ./blah.txt\n\nTruncat"
},
{
"path": "tests/list_file_types.t",
"chars": 5326,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n\nLanguage types are output:\n\n $ ag --list-file-types\n The following file types are sup"
},
{
"path": "tests/literal_word_regexp.t",
"chars": 757,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ echo 'blah abc def' > blah1.txt\n $ echo 'abc blah def' > blah2.txt\n $ echo 'abc de"
},
{
"path": "tests/max_count.t",
"chars": 913,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf \"blah\\n\" > blah.txt\n $ printf \"blah2\\n\" >> blah.txt\n $ printf \"blah2\\n\" > b"
},
{
"path": "tests/multiline.t",
"chars": 365,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'what\\n' > blah.txt\n $ printf 'ever\\n' >> blah.txt\n $ printf 'whatever\\n' >"
},
{
"path": "tests/negated_options.t",
"chars": 982,
"preview": "Setup:\n\n $ . \"${TESTDIR}/setup.sh\"\n\nShould accept both --no-<option> and --no<option> forms.\n\n(Here we're just parsing "
},
{
"path": "tests/one_device.t",
"chars": 671,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n > if [ ! -e \"/dev/shm\" ]; then\n > echo \"No /dev/shm. Skipping test.\"\n > exit 80\n > "
},
{
"path": "tests/only_matching.t",
"chars": 1048,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf \"the quick brown foxy\\n\" > blah.txt\n $ printf \"blah blah blah\\n\" >> blah.txt"
},
{
"path": "tests/option_g.t",
"chars": 152,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ touch foobar\n\nSearch for lines matching \"hello\" in test_vimgrep.txt:\n\n $ ag -g foob"
},
{
"path": "tests/option_smartcase.t",
"chars": 285,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'asdf\\n' > test.txt\n $ printf 'AsDf\\n' >> test.txt\n\nSmart case search:\n\n $ "
},
{
"path": "tests/passthrough.t",
"chars": 551,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ unalias ag\n $ alias ag=\"$TESTDIR/../ag --noaffinity --nocolor --workers=1\"\n $ prin"
},
{
"path": "tests/pipecontext.t",
"chars": 502,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir pipecontext_dir\n $ printf \"a\\nb\\nc\\n\" > pipecontext_test.txt\n $ cd pipeconte"
},
{
"path": "tests/print_all_files.t",
"chars": 226,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'foo\\n' > ./foo.txt\n $ printf 'bar\\n' > ./bar.txt\n $ printf 'baz\\n' > ./baz"
},
{
"path": "tests/print_end.t",
"chars": 140,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ cp $TESTDIR/print_end.txt .\n\nPrint match at the end of a file\n\n $ ag ergneq1\n prin"
},
{
"path": "tests/print_end.txt",
"chars": 26,
"preview": "ergneqergneqergneq\nergneq1"
},
{
"path": "tests/search_stdin.t",
"chars": 138,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'blah\\n' > ./blah.txt\n\nFeed blah.txt from stdin:\n\n $ ag 'blah' < ./blah.txt\n"
},
{
"path": "tests/setup.sh",
"chars": 422,
"preview": "#!/bin/bash\n\n# All cram tests should use this. Make sure that \"ag\" runs the version\n# of ag we just built, and make the "
},
{
"path": "tests/stupid_fnmatch.t.disabled",
"chars": 286,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ mkdir -p ./a/bomb\n $ printf 'whatever\\n' > ./a/bomb/foo.yml\n $ printf '*b/foo.yml\\"
},
{
"path": "tests/vimgrep.t",
"chars": 824,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ printf 'Hello, \"Hello, world\" programs output \"Hello, world\".\\n' > ./test_vimgrep.tx"
},
{
"path": "tests/word_regexp.t",
"chars": 187,
"preview": "Setup:\n\n $ . $TESTDIR/setup.sh\n $ echo 'foo' > blah.txt\n $ echo 'bar' >> blah.txt\n $ echo 'foobar' >> blah.txt\n\nWord"
},
{
"path": "the_silver_searcher.spec.in",
"chars": 2315,
"preview": "%define _bashcompdir %_sysconfdir/bash_completion.d\n%define _zshcompdir %{_datadir}/zsh/site-functions\n\nName:\t\tthe_silve"
}
]
About this extraction
This page contains the full source code of the ggreer/the_silver_searcher GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 95 files (338.7 KB), approximately 87.1k tokens, and a symbol index with 112 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.