Repository: akavel/up Branch: master Commit: 840f23c21d3e Files: 8 Total size: 49.9 KB Directory structure: gitextract_mhae01k0/ ├── .travis.yml ├── AUTHORS ├── LICENSE ├── README.md ├── go.mod ├── go.sum ├── up.go └── up_test.go ================================================ FILE CONTENTS ================================================ ================================================ FILE: .travis.yml ================================================ language: go go: - 1.x - master env: - GO111MODULE=on ================================================ FILE: AUTHORS ================================================ Please keep the contents of this file sorted alphabetically. Александр Крамарев Calum MacRae Ghislain Rodrigues Mateusz Czapliński Rohan Verma ================================================ FILE: LICENSE ================================================ Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ================================================ FILE: README.md ================================================ # up - the Ultimate Plumber **up** is the **Ultimate Plumber**, a tool for writing Linux pipes in a terminal-based UI interactively, with instant live preview of command results. The main **goal** of the Ultimate Plumber is to help **interactively and incrementally explore textual data** in Linux, by making it easier to quickly build complex pipelines, thanks to a **fast feedback loop**. This is achieved by boosting any typical **Linux text-processing utils** such as `grep`, `sort`, `cut`, `paste`, `awk`, `wc`, `perl`, etc., etc., by providing a quick, **interactive, scrollable preview** of their results. [![](up.gif)](https://asciinema.org/a/208538) ## Usage **[Download *up* for Linux](https://github.com/akavel/up/releases/latest/download/up)**   |   [ArchLinux](https://wiki.archlinux.org/index.php/Arch_User_Repository): [`aur/up`](https://aur.archlinux.org/packages/up/)   |   FreeBSD: [`pkg install up`](https://www.freshports.org/textproc/up)   |   macOS: [`brew install up`](https://formulae.brew.sh/formula/up)   |   [Other OSes](https://github.com/akavel/up/releases) To start using **up**, redirect any text-emitting command (or pipeline) into it — for example: $ lshw |& ./up then: - use ***PgUp/PgDn*** and ***Ctrl-[←]/Ctrl-[→]*** for basic browsing through the command output; - in the input box at the top of the screen, start **writing any bash pipeline**; then **press Enter to execute the command you typed**, and the Ultimate Plumber will immediately show you the output of the pipeline in the **scrollable window** below (replacing any earlier contents) - For example, you can try writing: `grep network -A2 | grep : | cut -d: -f2- | paste - -` — on my computer, after pressing *Enter*, the screen then shows the pipeline and a scrollable preview of its output like below: | grep network -A2 | grep : | cut -d: -f2- | paste - - Wireless interface Centrino Advanced-N 6235 Ethernet interface RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller - **WARNING: Please be careful when using it! It could be dangerous.** In particular, writing "rm" or "dd" into it could be like running around with a chainsaw. But you'd be careful writing "rm" anywhere in Linux anyway, no? - when you are satisfied with the result, you can **press *Ctrl-X* to exit** the Ultimate Plumber, and the command you built will be **written into `up1.sh` file** in the current working directory (or, if it already existed, `up2.sh`, etc., until 1000, based on [Shlemiel the Painter's algorithm](https://www.joelonsoftware.com/2001/12/11/back-to-basics/)). Alternatively, you can press ***Ctrl-C*** to quit without saving. - If the command you piped into *up* is long-running (in such case you will see a tilde `~` indicator character in the top-left corner of the screen, meaning that *up* is still waiting for more input), you may need to press ***Ctrl-S*** to temporarily freeze *up*'s input buffer (a freeze will be indicated by a `#` character in top-left corner), which will inject a fake EOF into the pipeline; otherwise, some commands in the pipeline may not print anything, waiting for full input (especially commands like `wc` or `sort`, but `grep`, `perl`, etc. may also show incomplete results). To unfreeze back, press ***Ctrl-Q***. ## Additional Notes - The pipeline is passed verbatim to a `bash -c` command, so any bash-isms should work. - The input buffer of the Ultimate Plumber is currently fixed at **40 MB**. If you reach this limit, a `+` character should get displayed in the top-left corner of the screen. (This is intended to be changed to a dynamically/manually growable buffer in a future version of *up*.) - **MacOSX support:** I don't have a Mac, thus I have no idea if it works on one. You are welcome to try, and also to send PRs. If you're interested in me providing some kind of official-like support for MacOSX, please consider trying to find a way to send me some usable-enough Mac computer. Please note I'm not trying to "take advantage" of you by this, as I'm actually not at all interested in achieving a Mac otherwise. (Also, trying to commit to this kind of support will be an extra burden and obligation on me. Knowing someone out there cares enough to do a fancy physical gesture would really help alleviate this.) If you're serious enough to consider this option, please contact me by email (mailto:czapkofan@gmail.com) or keybase (https://keybase.io/akavel), so that we could try to research possible ways to achieve this. Thanks for understanding! - **Prior art:** I was surprised no one seemed to write a similar tool before, that I could find. It should have been possible to write this since the dawn of Unix already, or earlier! And indeed, after I announced *up*, I got enough publicity that my attention was directed to one such earlier project already: **[Pipecut](http://pipecut.org/index.html)**. Looks interesting! You may like to check it too! (Thanks [@TronDD](https://lobste.rs/s/acpz00/up_tool_for_writing_linux_pipes_with#c_qxrgoa).) - **Other influences:** I don't remember the fact too well already, but I'm rather sure that this must have been inspired in big part by The Bret Victor's Talk(s). ## Future Ideas - I have quite a lot of ideas for further experimentation of development of *up*, including but not limited to: - [RIIR](https://rust-lang.org) (once I learn enough of Rust... at some point in future... maybe...) — esp. to hopefully make *up* be a smaller binary (and also to maybe finally learn some Rust); though I'm somewhat afraid if it might ossify the codebase and make harder to develop further..? ...but maybe actually converse?... - Maybe it could be made into an UI-less, RPC/REST/socket/text-driven service, like gocode or [Language Servers](https://langserver.org/), for integration with editors/IDEs (emacs? vim? VSCode?...) I'd be especially interested in eventually merging it into [Luna Studio](https://luna-lang.org/); RIIR may help in this. (Before this, as a simpler approach, multi-line editing may be needed, or at least left&right scrolling of the command editor input box. Also, some kind of jumping between words in the command line; readline's *Alt-b* & *Alt-f*?) - Make it possible to [capture output of already running processes](https://stackoverflow.com/a/19584979/98528)! (But maybe that could be better made as a separate, composable tool! In Rust?) - Adding tests... (ahem; see also [#1](https://github.com/akavel/up/issues/1)) ...also write `--help`... - Making it work on Windows, somehow[?](https://github.com/mattn/go-shellwords) Also, obviously, would be nice to have some CI infrastructure enabling porting it to MacOSX, BSDs, etc., etc... - Integration with [fzf](https://github.com/junegunn/fzf) and other TUI tools? I only have some vague thoughts and ideas about it as of now, not even sure how this could look like. - Adding more previews, for each `|` in the pipeline; also forking of pipelines, merging, feedback loops, and other mixing and matching (though I'd strongly prefer if [Luna](https://luna-lang.org) was to do it eventually). - If you are interested in financing my R&D work, contact me by email at: czapkofan@gmail.com, or [on keybase.io as akavel](https://keybase.io/akavel). I suppose I will probably be developing the Ultimate Plumber further anyway, but at this time it's purely a hobby project, with all the fun and risks this entails. — *Mateusz Czapliński* *October 2018* *PS. The UP logo was conceived and generously sponsored by [Thoai Nguyen](https://github.com/thoaionline) and [GPU Exchange](https://gpu.exchange/), with a helping hand from [Many Pixels](https://www.manypixels.co/).* ================================================ FILE: go.mod ================================================ module github.com/akavel/up go 1.14 require ( github.com/gdamore/tcell v1.4.0 github.com/mattn/go-isatty v0.0.3 github.com/mattn/go-runewidth v0.0.9 // indirect github.com/spf13/pflag v1.0.3 golang.org/x/sys v0.0.0-20201029080932-201ba4db2418 // indirect golang.org/x/text v0.3.4 // indirect ) ================================================ FILE: go.sum ================================================ github.com/gdamore/encoding v1.0.0 h1:+7OoQ1Bc6eTm5niUzBa0Ctsh6JbMW6Ra+YNuAtDBdko= github.com/gdamore/encoding v1.0.0/go.mod h1:alR0ol34c49FCSBLjhosxzcPHQbf2trDkoo5dl+VrEg= github.com/gdamore/tcell v1.4.0 h1:vUnHwJRvcPQa3tzi+0QI4U9JINXYJlOz9yiaiPQ2wMU= github.com/gdamore/tcell v1.4.0/go.mod h1:vxEiSDZdW3L+Uhjii9c3375IlDmR05bzxY404ZVSMo0= github.com/lucasb-eyer/go-colorful v1.0.3 h1:QIbQXiugsb+q10B+MI+7DI1oQLdmnep86tWFlaaUAac= github.com/lucasb-eyer/go-colorful v1.0.3/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0= github.com/mattn/go-isatty v0.0.3 h1:ns/ykhmWi7G9O+8a448SecJU3nSMBXJfqQkl0upE1jI= github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/mattn/go-runewidth v0.0.7 h1:Ei8KR0497xHyKJPAv59M1dkC+rOZCMBJ+t3fZ+twI54= github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0= github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg= github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= golang.org/x/sys v0.0.0-20190626150813-e07cf5db2756 h1:9nuHUbU8dRnRRfj9KjWUVrJeoexdbeMjttk6Oh1rD10= golang.org/x/sys v0.0.0-20190626150813-e07cf5db2756/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201029080932-201ba4db2418 h1:HlFl4V6pEMziuLXyRkm5BIYq1y1GAbb02pRlWvI54OM= golang.org/x/sys v0.0.0-20201029080932-201ba4db2418/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.4 h1:0YWbFKbhXG/wIiuHDSKpS0Iy7FSA+u45VtBMfQcFTTc= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= ================================================ FILE: up.go ================================================ // Copyright 2018 The up AUTHORS // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. // up is the Ultimate Plumber, a tool for writing Linux pipes in a // terminal-based UI interactively, with instant live preview of command // results. package main import ( "bufio" "bytes" "context" "crypto/sha1" "errors" "fmt" "io" "io/ioutil" "log" "os" "os/exec" "sync" "unicode" "github.com/gdamore/tcell" "github.com/gdamore/tcell/terminfo" "github.com/mattn/go-isatty" "github.com/spf13/pflag" ) const version = "0.4 (2020-10-29)" // TODO: in case of error, show it in red (bg?), then below show again initial normal output (see also #4) // TODO: F1 should display help, and it should be multi-line, and scrolling licensing credits // TODO: some key shortcut to increase stdin capture buffer size (unless EOF already reached) // TODO: show status infos: // - red fg + "up: process returned with error code %d" -- when subprocess returned an error // - yellow fg -- when process is still not finished // TODO: on github: add issues, incl. up-for-grabs / help-wanted // TODO: [LATER] make it work on Windows; maybe with mattn/go-shellwords ? // TODO: [LATER] Ctrl-O shows input via `less` or $PAGER // TODO: properly show all licenses of dependencies on --version // TODO: [LATER] on ^X (?), leave TUI and run the command through buffered input, then unpause rest of input // TODO: [LATER] allow adding more elements of pipeline (initially, just writing `foo | bar` should work) // TODO: [LATER] allow invocation with partial command, like: `up grep -i` (see also #11) // TODO: [LATER][MAYBE] allow reading upN.sh scripts (see also #11) // TODO: [MUCH LATER] readline-like rich editing support? and completion? (see also #28) // TODO: [MUCH LATER] integration with fzf? and pindexis/marker? // TODO: [LATER] forking and unforking pipelines (see also #4) // TODO: [LATER] capture output of a running process (see: https://stackoverflow.com/q/19584825/98528) // TODO: [LATER] richer TUI: // - show # of read lines & kbytes // - show status (errorlevel) of process, or that it's still running (also with background colors) // - allow copying and pasting to/from command line // TODO: [LATER] allow connecting external editor (become server/engine via e.g. socket) // TODO: [LATER] become pluggable into http://luna-lang.org // TODO: [LATER][MAYBE] allow "plugins" ("combos" - commands with default options) e.g. for Lua `lua -e`+auto-quote, etc. // TODO: [LATER] make it more friendly to infrequent Linux users by providing "descriptive" commands like "search" etc. // TODO: [LATER] advertise on some reddits for data exploration / data science // TODO: [LATER] undo/redo - history of commands (see also #4) // TODO: [LATER] jump between buffers saved from earlier pipe fragments; OR: allow saving/recalling "snapshots" of (cmd, results) pairs (see also #4) // TODO: [LATER] ^-, U -- to switch to "unsafe mode"? -u to switch back? + some visual marker func init() { pflag.Usage = func() { fmt.Fprint(os.Stderr, `Usage: COMMAND | up [OPTIONS] up is the Ultimate Plumber, a tool for writing Linux pipes in a terminal-based UI interactively, with instant live preview of command results. To start using up, redirect any text-emitting command (or pipeline) into it - for example: $ lshw |& ./up Ultimate Plumber then opens a full-screen terminal app. The top line of the screen can be edited in order to interactively build a pipeline. Every time you hit [Enter], the bottom of the screen will display the results of passing the up's standard input through the pipeline (executed using your default $SHELL). If a tilde '~' is visible in top-left corner, it indicates that Ultimate Plumber did not yet fully consume its input. Some pipelines may not finish with incomplete input; use Ctrl-S to freeze reading the input and to inject fake EOF; use Ctrl-Q to unfreeze back and continue reading. If a plus '+' is visible in top-left corner, the internal buffer limit (default: 40MB) was reached and Ultimate Plumber won't read more input. KEYS - alphanumeric & symbol keys, Left, Right, Ctrl-A/E/B/F/K/Y/W - navigate and edit the pipeline command - Enter - execute the pipeline command, updating the pipeline output panel - Up, Dn, PgUp, PgDn, Ctrl-Left, Ctrl-Right - navigate (scroll) the pipeline output panel - Ctrl-X - exit and write the pipeline to up1.sh (or if it exists then to up2.sh, etc. till up1000.sh) - Ctrl-C - quit without saving and emit the pipeline on standard output - Ctrl-S - temporarily freeze a long-running input to Ultimate Plumber, injecting a fake EOF into the buffer (shows '#' indicator in top-left corner) - Ctrl-Q - unfreeze back after Ctrl-S (disables '#' indicator) OPTIONS `) pflag.PrintDefaults() fmt.Fprint(os.Stderr, ` HOMEPAGE: https://github.com/akavel/up VERSION: `+version+` `) } pflag.ErrHelp = errors.New("") // TODO: or something else? } var ( // TODO: dangerous? immediate? raw? unsafe? ... // FIXME(akavel): mark the unsafe mode vs. safe mode with some colour or status; also inform/mark what command's results are displayed... unsafeMode = pflag.Bool("unsafe-full-throttle", false, "enable mode in which pipeline is executed immediately after any change (without pressing Enter)") outputScript = pflag.StringP("output-script", "o", "", "save the command to specified `file` if Ctrl-X is pressed (default: up.sh)") debugMode = pflag.Bool("debug", false, "debug mode") noColors = pflag.Bool("no-colors", false, "disable interface colors") shellFlag = pflag.StringArrayP("exec", "e", nil, "`command` to run pipeline with; repeat multiple times to pass multi-word command; defaults to '-e=$SHELL -e=-c'") initialCmd = pflag.StringP("pipeline", "c", "", "initial `commands` to use as pipeline (default empty)") bufsize = pflag.Int("buf", 40, "input buffer size & pipeline buffer sizes in `megabytes` (MiB)") noinput = pflag.Bool("noinput", false, "start with empty buffer regardless if any input was provided") ) func main() { // Handle command-line flags pflag.Parse() log.SetOutput(ioutil.Discard) if *debugMode { debug, err := os.Create("up.debug") if err != nil { die(err.Error()) } log.SetOutput(debug) } // Find out what is the user's preferred login shell. This also allows user // to choose the "engine" used for command execution. shell := *shellFlag if len(shell) == 0 { log.Println("checking $SHELL...") sh := os.Getenv("SHELL") if sh != "" { goto shell_found } log.Println("checking bash...") sh, _ = exec.LookPath("bash") if sh != "" { goto shell_found } log.Println("checking sh...") sh, _ = exec.LookPath("sh") if sh != "" { goto shell_found } die("cannot find shell: no -e flag, $SHELL is empty, neither bash nor sh are in $PATH") shell_found: shell = []string{sh, "-c"} } log.Println("found shell:", shell) stdin := io.Reader(os.Stdin) if *noinput { stdin = bytes.NewReader(nil) } else if isatty.IsTerminal(os.Stdin.Fd()) { // TODO: Without this block, we'd hang when nothing is piped on input (see // github.com/peco/peco, mattn/gof, fzf, etc.) die("up requires some data piped on standard input, for example try: `echo hello world | up`") } // Initialize TUI infrastructure tui := initTUI() defer tui.Fini() // Initialize 3 main UI parts var ( // The top line of the TUI is an editable command, which will be used // as a pipeline for data we read from stdin commandEditor = NewEditor("| ", *initialCmd) // The rest of the screen is a view of the results of the command commandOutput = BufView{} // Sometimes, a message may be displayed at the bottom of the screen, with help or other info message = `Enter runs ^X exit (^C nosave) PgUp/PgDn/Up/Dn/^ scroll ^S pause (^Q end) [Ultimate Plumber v` + version + ` by akavel et al.]` ) // Initialize main data flow var ( // We capture data piped to 'up' on standard input into an internal buffer // When some new data shows up on stdin, we raise a custom signal, // so that main loop will refresh the buffers and the output. stdinCapture = NewBuf(*bufsize*1024*1024). StartCapturing(stdin, func() { triggerRefresh(tui) }) // Then, we pass this data as input to a subprocess. // Initially, no subprocess is running, as no command is entered yet commandSubprocess *Subprocess = nil ) // Intially, for user's convenience, show the raw input data, as if `cat` command was typed commandOutput.Buf = stdinCapture // Main loop lastCommand := "" restart := false for { // If user edited the command, immediately run it in background, and // kill the previously running command. command := commandEditor.String() if restart || (*unsafeMode && command != lastCommand) { commandSubprocess.Kill() if command != "" { commandSubprocess = StartSubprocess(shell, command, stdinCapture, func() { triggerRefresh(tui) }) commandOutput.Buf = commandSubprocess.Buf } else { // If command is empty, show original input data again (~ equivalent of typing `cat`) commandSubprocess = nil commandOutput.Buf = stdinCapture } restart = false lastCommand = command } // Draw UI w, h := tui.Size() style := whiteOnBlue if command == lastCommand { style = whiteOnDBlue } stdinCapture.DrawStatus(TuiRegion(tui, 0, 0, 1, 1), style) commandEditor.DrawTo(TuiRegion(tui, 1, 0, w-1, 1), style, func(x, y int) { tui.ShowCursor(x+1, 0) }) commandOutput.DrawTo(TuiRegion(tui, 0, 1, w, h-1)) drawText(TuiRegion(tui, 0, h-1, w, 1), whiteOnBlue, message) tui.Show() // Handle UI events switch ev := tui.PollEvent().(type) { // Key pressed case *tcell.EventKey: // Is it a command editor key? if commandEditor.HandleKey(ev) { message = "" continue } // Is it a command output view key? if commandOutput.HandleKey(ev, h-1) { message = "" continue } // Some other global key combinations switch getKey(ev) { case key(tcell.KeyEnter): restart = true case key(tcell.KeyCtrlUnderscore), ctrlKey(tcell.KeyCtrlUnderscore): // TODO: ask for another character to trigger command-line option, like in `less` case key(tcell.KeyCtrlS), ctrlKey(tcell.KeyCtrlS): stdinCapture.Pause(true) triggerRefresh(tui) case key(tcell.KeyCtrlQ), ctrlKey(tcell.KeyCtrlQ): stdinCapture.Pause(false) restart = true case key(tcell.KeyCtrlC), ctrlKey(tcell.KeyCtrlC), key(tcell.KeyCtrlD), ctrlKey(tcell.KeyCtrlD): // Quit tui.Fini() os.Stderr.WriteString("up: Ultimate Plumber v" + version + " https://github.com/akavel/up\n") os.Stderr.WriteString("up: | " + commandEditor.String() + "\n") return case key(tcell.KeyCtrlX), ctrlKey(tcell.KeyCtrlX): // Write script 'upN.sh' and quit tui.Fini() writeScript(shell, commandEditor.String(), tui) return } } } } func initTUI() tcell.Screen { // TODO: maybe try gocui or termbox? tui, err := tcell.NewScreen() if err == terminfo.ErrTermNotFound { term := os.Getenv("TERM") hash := sha1.Sum([]byte(term)) // TODO: add a flag which would attempt to perform the download automatically if explicitly requested by user die(fmt.Sprintf(`%[1]s Your terminal code: TERM=%[2]s was not found in the database provided by tcell library. Please try checking if a supplemental database is found for your terminal at one of the following URLs: https://github.com/gdamore/tcell/raw/master/terminfo/database/%.1[3]x/%.4[3]x https://github.com/gdamore/tcell/raw/master/terminfo/database/%.1[3]x/%.4[3]x.gz If yes, download it and save in the following directory: $HOME/.tcelldb/%.1[3]x/ then try running "up" again. If that does not work for you, please first consult: https://github.com/akavel/up/issues/15 and if you don't see your terminal code mentioned there, please try asking on: https://github.com/gdamore/tcell/issues Or, you might try changing TERM temporarily to some other value, for example by running "up" with: TERM=xterm up Good luck!`, err, term, hash)) } if err != nil { die(err.Error()) } err = tui.Init() if err != nil { die(err.Error()) } return tui } func triggerRefresh(tui tcell.Screen) { tui.PostEvent(tcell.NewEventInterrupt(nil)) } func die(message string) { os.Stderr.WriteString("error: " + message + "\n") os.Exit(1) } func NewEditor(prompt, value string) *Editor { v := []rune(value) return &Editor{ prompt: []rune(prompt), value: v, cursor: len(v), lastw: len(v), } } type Editor struct { // TODO: make editor multiline. Reuse gocui or something for this? prompt []rune value []rune killspace []rune cursor int // lastw is length of value on last Draw; we need it to know how much to erase after backspace lastw int } func (e *Editor) String() string { return string(e.value) } func (e *Editor) DrawTo(region Region, style tcell.Style, setcursor func(x, y int)) { // Draw prompt & the edited value - use white letters on blue background for i, ch := range e.prompt { region.SetCell(i, 0, style, ch) } for i, ch := range e.value { region.SetCell(len(e.prompt)+i, 0, style, ch) } // Clear remains of last value if needed for i := len(e.value); i < e.lastw; i++ { region.SetCell(len(e.prompt)+i, 0, tcell.StyleDefault, ' ') } e.lastw = len(e.value) // Show cursor if requested if setcursor != nil { setcursor(len(e.prompt)+e.cursor, 0) } } func (e *Editor) HandleKey(ev *tcell.EventKey) bool { // If a character is entered, with no modifiers except maybe shift, then just insert it if ev.Key() == tcell.KeyRune && ev.Modifiers()&(^tcell.ModShift) == 0 { e.insert(ev.Rune()) return true } // Handle editing & movement keys switch getKey(ev) { case key(tcell.KeyBackspace), key(tcell.KeyBackspace2): // See https://github.com/nsf/termbox-go/issues/145 e.delete(-1) case key(tcell.KeyDelete): e.delete(0) case key(tcell.KeyLeft), key(tcell.KeyCtrlB), ctrlKey(tcell.KeyCtrlB): if e.cursor > 0 { e.cursor-- } case key(tcell.KeyRight), key(tcell.KeyCtrlF), ctrlKey(tcell.KeyCtrlF): if e.cursor < len(e.value) { e.cursor++ } case key(tcell.KeyCtrlA), ctrlKey(tcell.KeyCtrlA): e.cursor = 0 case key(tcell.KeyCtrlE), ctrlKey(tcell.KeyCtrlE): e.cursor = len(e.value) case key(tcell.KeyCtrlK), ctrlKey(tcell.KeyCtrlK): e.kill() case key(tcell.KeyCtrlY), ctrlKey(tcell.KeyCtrlY): e.insert(e.killspace...) case key(tcell.KeyCtrlW), ctrlKey(tcell.KeyCtrlW): e.unixWordRubout() default: // Unknown key/combination, not handled return false } return true } func (e *Editor) insert(ch ...rune) { // Based on https://github.com/golang/go/wiki/SliceTricks#insert e.value = append(e.value, ch...) // = PREFIX + SUFFIX + (filler) copy(e.value[e.cursor+len(ch):], e.value[e.cursor:]) // = PREFIX + (filler) + SUFFIX copy(e.value[e.cursor:], ch) // = PREFIX + ch + SUFFIX e.cursor += len(ch) } func (e *Editor) delete(dx int) { pos := e.cursor + dx if pos < 0 || pos >= len(e.value) { return } e.value = append(e.value[:pos], e.value[pos+1:]...) e.cursor = pos } func (e *Editor) kill() { if e.cursor != len(e.value) { e.killspace = append(e.killspace[:0], e.value[e.cursor:]...) } e.value = e.value[:e.cursor] } // unixWordRubout removes the part of the word on the left of the cursor. A word is // delimited by whitespaces. // The term `unix-word-rubout` comes from `readline` (see `man 3 readline`) func (e *Editor) unixWordRubout() { if e.cursor <= 0 { return } pos := e.cursor - 1 for pos != 0 && (unicode.IsSpace(e.value[pos]) || !unicode.IsSpace(e.value[pos-1])) { pos-- } e.killspace = append(e.killspace[:0], e.value[pos:e.cursor]...) e.value = append(e.value[:pos], e.value[e.cursor:]...) e.cursor = pos } type BufView struct { // TODO: Wrap bool Y int // Y of the view in the Buf, for down/up scrolling X int // X of the view in the Buf, for left/right scrolling Buf *Buf } func (v *BufView) DrawTo(region Region) { r := bufio.NewReader(v.Buf.NewReader(false)) // PgDn/PgUp etc. support for y := v.Y; y > 0; y-- { line, err := r.ReadBytes('\n') switch err { case nil: // skip line continue case io.EOF: r = bufio.NewReader(bytes.NewReader(line)) y = 0 break default: panic(err) } } lclip := false drawch := func(x, y int, ch rune) { if x <= v.X && v.X != 0 { x, ch = 0, '«' lclip = true } else { x -= v.X } if x >= region.W { x, ch = region.W-1, '»' } region.SetCell(x, y, tcell.StyleDefault, ch) } endline := func(x, y int) { x -= v.X if x < 0 { x = 0 } if x == 0 && lclip { x++ } lclip = false for ; x < region.W; x++ { region.SetCell(x, y, tcell.StyleDefault, ' ') } } x, y := 0, 0 // TODO: handle runes properly, including their visual width (mattn/go-runewidth) for { ch, _, err := r.ReadRune() if y >= region.H || err == io.EOF { break } else if err != nil { panic(err) } switch ch { case '\n': endline(x, y) x, y = 0, y+1 continue case '\t': const tabwidth = 8 drawch(x, y, ' ') for x%tabwidth < (tabwidth - 1) { x++ if x >= region.W { break } drawch(x, y, ' ') } default: drawch(x, y, ch) } x++ } for ; y < region.H; y++ { endline(x, y) x = 0 } } func (v *BufView) HandleKey(ev *tcell.EventKey, scrollY int) bool { const scrollX = 8 // When user scrolls horizontally, move by this many characters switch getKey(ev) { // // Vertical scrolling // case key(tcell.KeyUp): v.Y-- v.normalizeY() case key(tcell.KeyDown): v.Y++ v.normalizeY() case key(tcell.KeyPgDn): // TODO: in top-right corner of Buf area, draw current line number & total # of lines v.Y += scrollY v.normalizeY() case key(tcell.KeyPgUp): v.Y -= scrollY v.normalizeY() // // Horizontal scrolling // case altKey(tcell.KeyLeft), ctrlKey(tcell.KeyLeft): v.X -= scrollX if v.X < 0 { v.X = 0 } case altKey(tcell.KeyRight), ctrlKey(tcell.KeyRight): v.X += scrollX case altKey(tcell.KeyHome), ctrlKey(tcell.KeyHome): v.X = 0 default: // Unknown key/combination, not handled return false } return true } func (v *BufView) normalizeY() { nlines := count(v.Buf.NewReader(false), '\n') + 1 if v.Y >= nlines { v.Y = nlines - 1 } if v.Y < 0 { v.Y = 0 } } func count(r io.Reader, b byte) (n int) { buf := [256]byte{} for { i, err := r.Read(buf[:]) n += bytes.Count(buf[:i], []byte{b}) if err != nil { return } } } func NewBuf(bufsize int) *Buf { // TODO: make buffer size dynamic (growable by pressing a key) buf := &Buf{bytes: make([]byte, bufsize)} buf.cond = sync.NewCond(&buf.mu) return buf } type Buf struct { bytes []byte mu sync.Mutex // guards the following fields cond *sync.Cond status bufStatus n int } type bufStatus int const ( bufReading bufStatus = iota bufEOF bufPaused ) func (b *Buf) StartCapturing(r io.Reader, notify func()) *Buf { go b.capture(r, notify) return b } func (b *Buf) capture(r io.Reader, notify func()) { // TODO: allow stopping - take context? for { n, err := r.Read(b.bytes[b.n:]) b.mu.Lock() for b.status == bufPaused { b.cond.Wait() } b.n += n if err == io.EOF { b.status = bufEOF } if b.n == len(b.bytes) { // TODO: remove this when we can grow the buffer err = io.EOF } b.cond.Broadcast() b.mu.Unlock() go notify() if err == io.EOF { log.Printf("capture EOF after: %q", b.bytes[:b.n]) // TODO: make sure no race here, and skipped if not debugging return } else if err != nil { // TODO: better handling of errors panic(err) } } } func (b *Buf) Pause(pause bool) { b.mu.Lock() if pause { if b.status == bufReading { b.status = bufPaused // trigger all readers to emit fake EOF b.cond.Broadcast() } } else { if b.status == bufPaused { b.status = bufReading // wake up the capture func b.cond.Broadcast() } } b.mu.Unlock() } func (b *Buf) DrawStatus(region Region, style tcell.Style) { status := '~' // default: still reading input b.mu.Lock() switch { case b.status == bufPaused: status = '#' case b.status == bufEOF: status = ' ' // all input read, nothing more to do case b.n == len(b.bytes): status = '+' // buffer full } b.mu.Unlock() region.SetCell(0, 0, style, status) } func (b *Buf) NewReader(blocking bool) io.Reader { i := 0 return funcReader(func(p []byte) (n int, err error) { b.mu.Lock() end := b.n for blocking && end == i && b.status == bufReading && end < len(b.bytes) { b.cond.Wait() end = b.n } b.mu.Unlock() n = copy(p, b.bytes[i:end]) i += n if n > 0 { return n, nil } else { if blocking { log.Printf("blocking reader emitting EOF after: %q", b.bytes[:end]) } return 0, io.EOF } }) } type funcReader func([]byte) (int, error) func (f funcReader) Read(p []byte) (int, error) { return f(p) } type Subprocess struct { Buf *Buf cancel context.CancelFunc } func StartSubprocess(shell []string, command string, stdin *Buf, notify func()) *Subprocess { ctx, cancel := context.WithCancel(context.TODO()) r, w := io.Pipe() p := &Subprocess{ Buf: NewBuf(len(stdin.bytes)).StartCapturing(r, notify), cancel: cancel, } cmd := exec.CommandContext(ctx, shell[0], append(shell[1:], command)...) cmd.Stdout = w cmd.Stderr = w cmd.Stdin = stdin.NewReader(true) err := cmd.Start() if err != nil { fmt.Fprintf(w, "up: %s", err) w.Close() return p } log.Println(cmd.Path) go func() { err = cmd.Wait() if err != nil { fmt.Fprintf(w, "up: %s", err) log.Printf("Wait returned error: %s", err) } w.Close() }() return p } func (s *Subprocess) Kill() { if s == nil { return } s.cancel() } type key int32 func getKey(ev *tcell.EventKey) key { return key(ev.Modifiers())<<16 + key(ev.Key()) } func altKey(base tcell.Key) key { return key(tcell.ModAlt)<<16 + key(base) } func ctrlKey(base tcell.Key) key { return key(tcell.ModCtrl)<<16 + key(base) } func writeScript(shell []string, command string, tui tcell.Screen) { os.Stderr.WriteString("up: Ultimate Plumber v" + version + " https://github.com/akavel/up\n") var f *os.File var err error if *outputScript != "" { os.Stderr.WriteString("up: writing " + *outputScript) f, err = os.OpenFile(*outputScript, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0755) if err != nil { goto fallback_tmp } goto try_file } os.Stderr.WriteString("up: writing: .") for i := 1; i < 1000; i++ { f, err = os.OpenFile(fmt.Sprintf("up%d.sh", i), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0755) switch { case os.IsExist(err): continue case err != nil: goto fallback_tmp default: os.Stderr.WriteString("/" + f.Name()) goto try_file } } os.Stderr.WriteString(" - error: up1.sh-up999.sh already exist\n") goto fallback_tmp try_file: // NOTE: currently not supporting multi-word shell in upNNN.sh unfortunately :( _, err = fmt.Fprintf(f, "#!%s\n%s\n", shell[0], command) if err != nil { goto fallback_tmp } err = f.Close() if err != nil { goto fallback_tmp } os.Stderr.WriteString(" - OK\n") return fallback_tmp: // TODO: test if the fallbacks etc. protections actually work os.Stderr.WriteString(" - error: " + err.Error() + "\n") f, err = ioutil.TempFile("", "up-*.sh") if err != nil { goto fallback_print } _, err = fmt.Fprintf(f, "#!%s\n%s\n", shell, command) if err != nil { goto fallback_print } err = f.Close() if err != nil { goto fallback_print } os.Stderr.WriteString("up: writing: " + f.Name() + " - OK\n") os.Chmod(f.Name(), 0755) return fallback_print: fname := "TMP" if f != nil { fname = f.Name() } os.Stderr.WriteString("up: writing: " + fname + " - error: " + err.Error() + "\n") os.Stderr.WriteString("up: | " + command + "\n") } type Region struct { W, H int SetCell func(x, y int, style tcell.Style, ch rune) } func TuiRegion(tui tcell.Screen, x, y, w, h int) Region { return Region{ W: w, H: h, SetCell: func(dx, dy int, style tcell.Style, ch rune) { if dx >= 0 && dx < w && dy >= 0 && dy < h { if *noColors { style = tcell.StyleDefault } tui.SetCell(x+dx, y+dy, style, ch) } }, } } var ( whiteOnBlue = tcell.StyleDefault.Foreground(tcell.ColorWhite).Background(tcell.ColorBlue) whiteOnDBlue = tcell.StyleDefault.Foreground(tcell.ColorWhite).Background(tcell.ColorNavy) ) func drawText(region Region, style tcell.Style, text string) { for x, ch := range text { region.SetCell(x, 0, style, ch) } } ================================================ FILE: up_test.go ================================================ package main import "testing" func Test_Editor_insert(t *testing.T) { tests := []struct { comment string e Editor insert []rune wantValue []rune }{ { comment: "prepend ASCII char", e: Editor{ value: []rune(`abc`), cursor: 0, }, insert: []rune{'X'}, wantValue: []rune(`Xabc`), }, { comment: "prepend UTF char", e: Editor{ value: []rune(`abc`), cursor: 0, }, insert: []rune{'☃'}, wantValue: []rune(`☃abc`), }, { comment: "insert ASCII char", e: Editor{ value: []rune(`abc`), cursor: 1, }, insert: []rune{'X'}, wantValue: []rune(`aXbc`), }, { comment: "insert UTF char", e: Editor{ value: []rune(`abc`), cursor: 1, }, insert: []rune{'☃'}, wantValue: []rune(`a☃bc`), }, { comment: "append ASCII char", e: Editor{ value: []rune(`abc`), cursor: 3, }, insert: []rune{'X'}, wantValue: []rune(`abcX`), }, { comment: "append UTF char", e: Editor{ value: []rune(`abc`), cursor: 3, }, insert: []rune{'☃'}, wantValue: []rune(`abc☃`), }, { comment: "insert 2 ASCII chars", e: Editor{ value: []rune(`abc`), cursor: 1, }, insert: []rune{'X', 'Y'}, wantValue: []rune(`aXYbc`), }, } for _, tt := range tests { tt.e.insert(tt.insert...) if string(tt.e.value) != string(tt.wantValue) { t.Errorf("%q: bad value\nwant: %q\nhave: %q", tt.comment, tt.wantValue, tt.e.value) } } } func Test_Editor_unix_word_rubout(t *testing.T) { tests := []struct { comment string e Editor wantValue []rune wantKillspace []rune }{ { comment: "unix-word-rubout at beginning of line", e: Editor{ value: []rune(`abc`), cursor: 0, }, wantValue: []rune(`abc`), wantKillspace: []rune(``), }, { comment: "unix-word-rubout at soft beginning of line", e: Editor{ value: []rune(` abc`), cursor: 1, }, wantValue: []rune(`abc`), wantKillspace: []rune(` `), }, { comment: "unix-word-rubout until soft beginning of line", e: Editor{ value: []rune(` abc`), cursor: 2, }, wantValue: []rune(` bc`), wantKillspace: []rune(`a`), }, { comment: "unix-word-rubout until beginning of line", e: Editor{ value: []rune(`abc`), cursor: 2, }, wantValue: []rune(`c`), wantKillspace: []rune(`ab`), }, { comment: "unix-word-rubout in middle of line", e: Editor{ value: []rune(`lorem ipsum dolor`), cursor: 11, }, wantValue: []rune(`lorem dolor`), wantKillspace: []rune(`ipsum`), }, { comment: "unix-word-rubout cursor at beginning of word", e: Editor{ value: []rune(`lorem ipsum dolor`), cursor: 12, }, wantValue: []rune(`lorem dolor`), wantKillspace: []rune(`ipsum `), }, { comment: "unix-word-rubout cursor between multiple spaces", e: Editor{ value: []rune(`a b c`), cursor: 5, }, wantValue: []rune(`a c`), wantKillspace: []rune(`b `), }, { comment: "unix-word-rubout tab as space char (although is it a realistic case in the context of a command line instruction?)", e: Editor{ value: []rune(`a b c`), cursor: 5, }, wantValue: []rune(`a c`), wantKillspace: []rune(`b `), }, } for _, tt := range tests { tt.e.unixWordRubout() if string(tt.e.value) != string(tt.wantValue) { t.Errorf("%q: bad value\nwant: %q\nhave: %q", tt.comment, tt.wantValue, tt.e.value) } if string(tt.e.killspace) != string(tt.wantKillspace) { t.Errorf("%q: bad value in killspace\nwant: %q\nhave: %q", tt.comment, tt.wantKillspace, tt.e.value) } } }