Repository: jpmonettas/flow-storm-debugger
Branch: master
Commit: e64bd0cac4b3
Files: 112
Total size: 1.2 MB
Directory structure:
gitextract_xzq9jm4n/
├── .dir-locals.el
├── .github/
│ └── FUNDING.yml
├── .gitignore
├── CHANGELOG.md
├── Makefile
├── Readme.md
├── UNLICENSE
├── build.clj
├── deps.edn
├── docs/
│ ├── _config.yml
│ ├── dev_notes.md
│ ├── high_level_diagram.drawio
│ ├── index.html
│ ├── related-research-and-tools.md
│ ├── run_configs.drawio
│ ├── test-cases.org
│ ├── timeline.drawio
│ ├── user_guide.adoc
│ └── user_guide.html
├── examples/
│ └── plugins/
│ └── basic-plugin/
│ ├── deps.edn
│ └── src/
│ └── flow_storm/
│ └── plugins/
│ └── timelines_counters/
│ ├── all.clj
│ ├── runtime.cljc
│ └── ui.clj
├── llm-prompt.txt
├── package.json
├── resources/
│ └── flowstorm/
│ ├── fonts/
│ │ └── LICENSE.txt
│ └── styles/
│ ├── font-size-lg.css
│ ├── font-size-md.css
│ ├── font-size-sm.css
│ ├── font-size-xl.css
│ ├── styles.css
│ ├── theme_dark.css
│ └── theme_light.css
├── scripts/
│ ├── flow-clj
│ ├── gsettings
│ └── mock-gnome.sh
├── shadow-cljs.edn
├── src-dbg/
│ └── flow_storm/
│ └── debugger/
│ ├── docs.clj
│ ├── events_processor.clj
│ ├── events_queue.clj
│ ├── main.clj
│ ├── repl/
│ │ ├── core.clj
│ │ └── nrepl.clj
│ ├── runtime_api.clj
│ ├── state.clj
│ ├── tutorials/
│ │ └── basics.clj
│ ├── ui/
│ │ ├── browser/
│ │ │ └── screen.clj
│ │ ├── commons.clj
│ │ ├── components.clj
│ │ ├── data_windows/
│ │ │ ├── data_windows.clj
│ │ │ ├── visualizers/
│ │ │ │ └── oscilloscope.clj
│ │ │ └── visualizers.clj
│ │ ├── docs/
│ │ │ └── screen.clj
│ │ ├── flows/
│ │ │ ├── bookmarks.clj
│ │ │ ├── call_tree.clj
│ │ │ ├── code.clj
│ │ │ ├── components.clj
│ │ │ ├── functions.clj
│ │ │ ├── general.clj
│ │ │ ├── multi_thread_timeline.clj
│ │ │ ├── printer.clj
│ │ │ ├── screen.clj
│ │ │ └── search.clj
│ │ ├── main.clj
│ │ ├── outputs/
│ │ │ └── screen.clj
│ │ ├── plugins.clj
│ │ ├── tasks.clj
│ │ └── utils.clj
│ ├── user_guide.clj
│ └── websocket.clj
├── src-dev/
│ ├── dev.clj
│ ├── dev_tester.clj
│ ├── dev_tester.cljs
│ ├── dev_tester_12.clj
│ ├── logging.properties
│ └── user.clj
├── src-inst/
│ ├── data_readers.clj
│ └── flow_storm/
│ ├── api.clj
│ ├── api.cljs
│ ├── jobs.cljc
│ ├── nrepl/
│ │ └── middleware.clj
│ ├── ns_reload_utils.clj
│ ├── preload.cljs
│ ├── remote_websocket_client.clj
│ ├── remote_websocket_client.cljs
│ ├── runtime/
│ │ ├── debuggers_api.cljc
│ │ ├── events.cljc
│ │ ├── indexes/
│ │ │ ├── api.cljc
│ │ │ ├── form_registry.cljc
│ │ │ ├── protocols.cljc
│ │ │ ├── storm_form_registry.clj
│ │ │ ├── thread_registry.cljc
│ │ │ ├── timeline_index.cljc
│ │ │ ├── total_order_timeline.cljc
│ │ │ └── utils.cljc
│ │ ├── outputs.cljc
│ │ ├── types/
│ │ │ ├── bind_trace.cljc
│ │ │ ├── expr_trace.cljc
│ │ │ ├── fn_call_trace.cljc
│ │ │ └── fn_return_trace.cljc
│ │ └── values.cljc
│ ├── storm_api.clj
│ ├── storm_preload.cljs
│ └── tracer.cljc
├── src-shared/
│ └── flow_storm/
│ ├── eql.cljc
│ ├── form_pprinter.clj
│ ├── json_serializer.clj
│ ├── json_serializer.cljs
│ ├── state_management.cljc
│ ├── types.cljc
│ └── utils.cljc
└── tests.edn
================================================
FILE CONTENTS
================================================
================================================
FILE: .dir-locals.el
================================================
((clojure-mode . ((cider-clojure-cli-aliases . "dev:dev-tools:storm")
(clojure-dev-menu-name . "flow-storm-dev-menu")
(cider-jack-in-nrepl-middlewares . ("refactor-nrepl.middleware/wrap-refactor"
"cider.nrepl/cider-middleware"
"flow-storm.nrepl.middleware/wrap-flow-storm")))))
================================================
FILE: .github/FUNDING.yml
================================================
# These are supported funding model platforms
github: [jpmonettas]
# patreon: # Replace with a single Patreon username
# open_collective: # Replace with a single Open Collective username
# ko_fi: # Replace with a single Ko-fi username
# tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
# community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
# liberapay: # Replace with a single Liberapay username
# issuehunt: # Replace with a single IssueHunt username
# otechie: # Replace with a single Otechie username
# lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
# custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
================================================
FILE: .gitignore
================================================
.clj-kondo
target
.cpcache
.nrepl-port
.shadow-cljs
.cljs_node_repl
out/
node_modules
public/js/
/.calva/
/.lsp/
================================================
FILE: CHANGELOG.md
================================================
# Changelog
## master (unreleased)
### New Features
### Changes
### Bugs fixed
- Fix multi thread timeline viewer running out of colors
- Fix flow-storm.runtime.indexes/detailed-total-order-timeline
## 4.5.9 (28-10-2025)
### New Features
### Changes
- Improving flow-storm.api/prob-ref
### Bugs fixed
- Fixing oscilloscope
## 4.5.8 (13-10-2025)
### New Features
### Changes
### Bugs fixed
- Fix seqable visualizer when clicking on empty rows
## 4.5.7 (03-10-2025)
### New Features
### Changes
- Use java.util.logging to log, so it can be disabled with logging.properties
### Bugs fixed
## 4.5.6 (24-09-2025)
### New Features
### Changes
- Upgrading hansel to 0.1.90 to improve objects methods names tracing
### Bugs fixed
## 4.5.5 (10-09-2025)
### New Features
### Changes
### Bugs fixed
- Fix .nrepl-port not found exception
## 4.5.4 (9-09-2025)
### New Features
### Changes
### Bugs fixed
- Fix heap reporting for ClojureScript (where heap data is currently missing)
## 4.5.3 (02-09-2025)
### New Features
- When starting the ui for remote debugging, when port not provided check .nrepl-port
### Changes
- Update hansel to 0.1.87
### Bugs fixed
## 4.5.2 (01-09-2025)
### New Features
- Added JVM option flowstorm.heapLimit which can be set also from the UI to automatically stop recording when
the heap reaches the set size in megabytes
### Changes
- JVM option flowstorm.throwOnTraceLimit has been renamed to flowstorm.throwOnLimit
- Update hansel to 0.1.85
### Bugs fixed
- Fix exceptions instrumentation in vanilla for non #trace ones
## 4.5.1 (26-08-2025)
### New Features
- Data Windows on-create can be called with :preferred-size :small
- Adds oscilloscope support for DW :preferred-size :small
- Add :pre-require to flow-storm.debugger.main/start-debugger for easier data window visualizers in ClojureScript
### Changes
- Perf improvement: reduce garbage created by keep-timeline-sub-range
- Perf improvement: improve how we send navs-refs for datawindows which are nil most of the time
### Bugs fixed
## 4.5.0 (05-08-2025)
### New Features
- A better oscilloscope data window
### Changes
- Limit the exceptions we are tracking on the UI to 100 so we don't make the UI super slow when too many exceptions are fired
- Improve quick jump box for same function fired on many threads
### Bugs fixed
- Fixed transients inspection
## 4.4.6 (10-06-2026)
### New Features
- Added llm-prompt.txt that can be used to instruct LLMs on how to use FlowStorm
- Added utilities api functions useful for llm-prompt.txt
### Changes
### Bugs fixed
## 4.4.5 (06-06-2025)
### New Features
### Changes
### Bugs fixed
- Fix shallow-indexed aspect extractor
## 4.4.4 (26-05-2025)
### New Features
### Changes
### Bugs fixed
- Report interruptible task failed messages
- Don't swallow the exception on :dbg
## 4.4.3 (21-05-2025)
### New Features
### Changes
### Bugs fixed
- Fix data windows default visualizers system
## 4.4.2 (13-05-2025)
### New Features
### Changes
- Make :preview the default visualizer for numbers
- Improved thread breakpoints UX/UI
- All api calls from UI have now a timeout, configurable via flowstorm.uiTimeoutMillis
### Bugs fixed
- Fix fn calls list styling
- Fix Browser's function 'Break' button styling
## 4.4.1 (03-05-2025)
### New Features
### Changes
- Locals context menu changed "Define all frame vars" to "Define all" which only defines the visible ones.
This is to make the future less confusing, specially in the presence of loops.
- Sort locals by symbol name
### Bugs fixed
- Fix locals display inside loops
## 4.4.0 (17-04-2025)
### New Features
- Check for `:flow-storm.power-step/skip` value when power stepping
### Changes
- Internal refactor
### Bugs fixed
- Extra styles should be applied at the end
## 4.3.0 (29-03-2025)
### New Features
### Changes
- Reduced memory footprint by ~10% (jvm17) by removing this-idx from FnCall, ExprExec, FnReturn, FnUnwind entries
- Programmable API BREAKING:
- removed indexes-api/entry-idx now that each entry doesn't know its own index
- (indexes-api/get-fn-call timeline entry) -> (indexes-api/get-fn-call timeline idx)
- (indexes-api/get-sub-form timeline entry) -> (indexes-api/get-sub-form timeline idx)
### Bugs fixed
## 4.2.2 (18-03-2025)
### New Features
- Add flowstorm.autoUpdateUI jvm opt
- Add flowstorm.threadTraceLimit jvm opt
- Add flowstorm.throwOnThreadLimit jvm opt
- Add flowstorm.callTreeUpdate jvm opt
### Changes
- Disable eql-query-pprint by default
- Perf improvements. We were calling val-pprint in a bunch of places where v-ref :val-preview could be used
### Bugs fixed
- Catch exceptions on preview build when print fails
## 4.2.1 (11-03-2025)
### New Features
- Accept multiple flowstorm.plugins.namespaces.* jvm props that will get merged
### Changes
- Make the locals pane a table, so columns can be resized and searched
### Bugs fixed
## 4.2.0 (25-02-2025)
### New Features
- Basic plugins system
- Enable/disable instrumentation from the UI
### Changes
- Disable preview pprints by default for speed. You can still enable them via the Config menu.
### Bugs fixed
## 4.1.2 (11-02-2025)
### New Features
### Changes
- Make scope visualizer a "rolling scope"
- Show tree-view childs truncation message inside the tree-view instead of a popup alert
- Upgrade to hansel 0.1.84 which doesn't depend on core.async
### Bugs fixed
## 4.1.1 (29-01-2025)
### New Features
- Configurable pprint previews menu for slow to pprint values cases
- Very crude zoom-in/zoom-out on scope visualizer
### Changes
- Make *out* and *err* print panel font monospaced
- Make printer support printing multi-line strings when printing strings
### Bugs fixed
- Fix printer removing after flow cleanning
- Fix data-window eql extractor so it doesn't break on infinite sequences
## 4.1.0 (01-01-2025)
### New Features
- Add configurable auto jump to exceptions
- New eql-query-pprint visualizer
- New webpage visualizer
- Add support for setting the visualizer via flow-storm.api/data-window-push-val
- Add debugger/bookmark to set bookmarks from code and also quickly jump to the first one
- Enable multiple ClojureScript runtimes <> multiple debuggers via "flowstorm_ws_port"
### Changes
- Configurable thread tab auto-update every 1sec (on by default)
- Remove middleware dependency on cider-nrepl-middleware
- Aspect extractors signature deprecation. Now it recieves the object and a map with extras
### Bugs fixed
- Fix taps "search value on flows"
- Fix middleware not working with nrepl > 1.3.0-beta2
- Fix middleware for ClojureScript
- Fix identity and equality powerstepping for unwinds
## 4.0.2 (25-11-2024)
### New Features
### Changes
- Change flow exceptions combo to only show exception root instead of all unwinds
### Bugs fixed
## 4.0.1 (12-11-2024)
### New Features
### Changes
### Bugs fixed
- Fix middleware for Cider Storm
## 4.0.0 (11-11-2024)
### New Features
- DataWindow system
- Outputs tool (Clojure only)
### Changes
- Improved flow search for DataWindows support
- Update javafx to "21.0.4-ea+1"
### Bugs fixed
- Fix Quick Jump on multiple flows
- Fix UI displaying of multi line strings in varios places
- Fix printers enable state on printers screen open
## 3.17.4 (24-09-2024)
### New Features
### Changes
### Bugs fixed
- Fix thread trace count not updating on thread tab refresh
- Fix ctx menu showing on forms background with left click
- For ns reload functionality don't read forms with *read-eval* false
## 3.17.3 (27-08-2024)
### New Features
### Changes
- Use quoted-string-split on editor open commands
### Bugs fixed
## 3.17.2 (06-08-2024)
### New Features
### Changes
- Improved multi-thread timeline colors
- Do not deref all derefables automatically, just atoms, refs, agents and vars and pending realized ones. Leave the reast to snapshot-value system
### Bugs fixed
- Do not deref delays when tracing values
## 3.17.1 (26-07-2024)
### New Features
- Add before-reload and after-reload hooks for the namespace reloading utilities
### Changes
- Update tutorial
### Bugs fixed
- Improve ns auto reload for namespaces loaded with load-file
## 3.17.0 (23-07-2024)
### New Features
- Make Search, Printer and Multi-thread-timeline tools work per flow
- Pake Printer use the multi-thread timeline if available to allow for thread interleaving print debugging
- New fn-call power stepper
- Optional automatic namespaces reload after changing prefixes for storm
### Changes
- Flows UI refactor
- Make stack click jump to beg of the fn instead of prev step
### Bugs fixed
## 3.16.0 (03-07-2024)
### New Features
- Add printer support to print on all threads
- Add printers transform expression (Clojure only)
### Changes
- UI refactor for toolbars
- UI refactor for printers and multi-thread timeline
- Set a UI limit of 200 on the Exceptions menu entries to improve UI responsiveness under deep recursions exceptions.
### Bugs fixed
- Fix #181 close children windows if main window closed
- Fix out of boundaries on step-prev-over and step-next-over
## 3.15.8 (21-06-2024)
### New Features
### Changes
- Add recorded threads counter on the threads menu button
- Clean initialization console logs for ClojureScript
### Bugs fixed
- Fix remote debugging for vanilla
- Fix windows paths handling in middleware
## 3.15.7 (13-06-2024)
### New Features
### Changes
### Bugs fixed
- Fix FlowStorm not starting in OSX because of wrong taskbar icon url
## 3.15.6 (13-06-2024)
### New Features
- Add jump to first and last occurrences buttons to power stepping controls
### Changes
- #rtrace now clears the current recording flow before running again
- Upgrade ikonli-javafx to 12.3.1
- Add FlowStorm icon to the toolbar and taskbar
- Namespace FlowStorm resources so they don't collide with other resources
### Bugs fixed
- Fix thread-id lost from thread tab after tab refresh
- Fix running with nrepl >= 1.2
## 3.15.5 (11-05-2024)
### New Features
### Changes
### Bugs fixed
- Make the code stepper the default tab instead of the tree
## 3.15.4 (06-05-2024)
### New Features
### Changes
### Bugs fixed
- Fix auto tab switch on code jump
## 3.15.3 (02-05-2024)
### New Features
- Add Goto file:line on the menu
- Show open in editor for any form that contains line meta
### Changes
- Show value inspector `def` on the same screen
- Remove unnamed anonymous functions from Quick Jump
- Remove docs builder (now on dyna-spec)
- Remove tools.build as a dependency
- Make the coed stepper the default tab instead of the tree
### Bugs fixed
## 3.15.2 (18-04-2024)
### New Features
- Timelines notification updates and thread refresh button
### Changes
### Bugs fixed
## 3.15.1 (16-04-2024)
### New Features
### Changes
### Bugs fixed
- Don't run #rtrace when recording is paused
- Fix vanilla-instrument-var for clojurescript
- Fix #rtrace for clojurescript
## 3.15.0 (16-04-2024)
### New Features
- Implement storm instrumentation management in the browser
- Add "Copy qualified function symbol" and "Copy function calling form" to form menu
- Add only-functions? to multi-thread timeline
### Changes
- The record flow selector combo is now next to the tabs
- The threads selector is now a menu button instead of a list view to gain some UI space
- Automatically open first recorded thread
- Improved graphical tutorial
- Help menu with tutorial and user's guide
- Don't start recording by default
### Bugs fixed
## 3.14.0 (30-03-2024)
### New Features
- Add support to open forms in editors
- Add an option to set threads limit to throw
- There is a new, much more powerfull global search system instead of the old per timeline search.
- An improved Flows system that allows the user to record into multiple flows.
- Implement an improved loop navigation system
### Changes
- Improve pprint panes. Now all of the also show exceptions
- #rtrace0 ... #rtrace5 reader tags were removed since they aren't needed anymore with the new flow system
### Bugs fixed
- Theme dialog panes
- Fix inspector power stepper
## 3.13.1 (21-03-2024)
### New Features
### Changes
- Bring back the clear recordings button to the toolbar
- A faster search system for ClojureScript (for every functionality that searches on the timeline)
### Bugs fixed
## 3.13.0 (19-03-2024)
### New Features
- Implemented thread trace limit as a fuse for infinite loops/recursion
- All possibly expensive slow operations are now cancellable. This includes :
- List functions calls
- List prints with the printer
- Values search
- Multi-thread timeline
- All power stepping tools
- Quick jump
- All functions that collect from the timeline will report as they run, no need to wait to the end. This includes :
- List functions calls
- List prints with the printer
- Multi-thread timeline
- Add a menu bar to help with discoverability
- Add functions calls ret render checkbox that update on change
### Changes
- Change Exceptions from a ComboBox to MenuButton
- Functions calls auto update on selection
### Bugs fixed
## 3.12.2 (05-03-2024)
### New Features
### Changes
### Bugs fixed
- Fix functions lists for nil returns
## 3.12.1 (05-03-2024)
### New Features
### Changes
- Improved functions pane.
- Change stack double click behavior as per #150
- Display bookmarks and inspector windows centered on main window
- Display all dialogs centered on main window
### Bugs fixed
## 3.12.0 (15-02-2024)
### New Features
- Add identity-other-thread power stepper
### Changes
- Big refactor with a ton of improvements to the repl API
### Bugs fixed
## 3.11.2 (13-02-2024)
### New Features
### Changes
### Bugs fixed
- Fix quick jump
## 3.11.1 (08-02-2024)
### New Features
### Changes
- Better execution highlighting system
- Browser and functions calls list views update as they change with the keyboard
### Bugs fixed
- Fix multi-thread timeline search
- Close context menu if there is one already open
## 3.11.0 (01-02-2024)
### New Features
- Add exceptions display
### Changes
- :ex captured exception system removed - superseded by unwind tracing
### Bugs fixed
- [IMPORTANT!] Fix timeline structure when functions unwind (requires ClojureStorm >= 1.11.1-19)
- Fix value inspector stack showing val instead of key
## 3.10.0 (29-01-2024)
### New Features
- Add a timeline index column to the bookmarks table
- Add a thread timeline index column to the multi-thread timeline
- New same-coord and custom-same-coord power steppers
- New context menu to find recordings from unhighlighted text ("Jump forward here", etc)
### Changes
- Make tooltips have a 400ms delay instead of default 1000ms
- Display timeline indexes zero based (used to be 1 based) to be less confusing when using the repl api
- Add thread-ids everywhere thread-names show
- Show form line next to form namespace if known
- New stepper buttong layout
- Centralize bookmarks system (one bookmark system for all flows and threads)
- Step over doesn't stop at function boundaries anymore
- Improved value inspector performance on remote runtimes (via :val-preview)
### Bugs fixed
- Pasting in quick jump box doesn't fire autocomplete
- Quick jump should create and/or focus the tab
- Fix tree view freezes when a node contains too many childs
- Fix quick-jump on Clojure remote
## 3.9.1 (12-01-2024)
### New Features
- Add print-wrap controls on panes
### Changes
- Don't automatically switch to the Taps tab on new taps
### Bugs fixed
- Don't trace false as nil
- Update hansel to 0.1.81 to fix browser var instrumentation in cljc
## 3.9.0 (19-12-2023)
### New Features
- Add a bookmarking system
- Add navigation undo/redo system
### Changes
- Upgrade JavaFX to 21.0.1
- Push minimal supported JDK version to 17 with a path for using it with 11
- Improve keyboard event handling system to support different layouts
- Improve following current selected expression
- Remove double scrolling in code panes
- Make code stack pane jump a double-click
- Support multiple debugger instances running at the same time. Useful for debugging multiple build in cljs.
### Bugs fixed
- Fix "Add to prints" not showing on Vanilla
- Enter on time box focus the code
## 3.8.6 (17-11-2023)
### New Features
### Changes
- Don't print handled exception error messages on std-err since it messes up cider
### Bugs fixed
- Capture exceptions on cljs remote connect
- [Remote] Don't crash the debugger if there is an exception initializing the RT through the repl
## 3.8.5 (10-11-2023)
### New Features
### Changes
### Bugs fixed
- Patch for clojure.pprint bug https://ask.clojure.org/index.php/13455/clojure-pprint-pprint-bug-when-using-the-code-dispatch-table
## 3.8.4 (09-11-2023)
### New Features
- Ctrl-f copies the current qualified funcion symbol to the clipboard
- Ctrl-Shift-f copies the current function call form
- Right clicking on a tree node now shows "Copy qualified function symbol"
- Right clicking on a tree node now shows "Copy function calling form"
### Changes
- Big codebase refactor for make it cleaner
- Improved search functionality (faster and with better UX)
### Bugs fixed
- Fix functions list not showing entire functions names when they are large
## 3.8.3 (25-10-2023)
### New Features
- Add dynamic font inc/dec and theme rotation
### Changes
### Bugs fixed
## 3.8.2 (23-10-2023)
### New Features
### Changes
- Upgrading j-system-theme-detector to 3.8.1 to fix a NPE
- Downgrading JavaFX to 19.0.2 since >20 needs JDK>=17 and we still want JDK11
### Bugs fixed
## 3.8.1 (19-10-2023)
### New Features
### Changes
- Improved code highlighter. Replaces JavaFX standard TextFlow with RichTextFx CodeArea for improved performance.
- Change hansel to com.github.flow-storm/hansel 0.1.79 for the organization move
### Bugs fixed
- Fix #98 Stepping over big forms is very slow
## 3.7.5 (02-10-2023)
### New Features
- Add function call limits
### Changes
- Disable functionality that doesn't make sense under storm when working under ClojureStorm or ClojureScriptStorm
- Improve Printer thread selection so you don't need to constantly re-select thread
### Bugs fixed
## 3.7.4 (27-09-2023)
### New Features
- Add flow-storm.storm-preload for ClojureScriptStorm
### Changes
- Improved initialization system for remote debugging
- Reuduce callstack tree nodes args print level and depth for perf (specially on remotes)
### Bugs fixed
## 3.7.3 (10-09-2023)
### New Features
- Add multimethod dispatch-val to stack pane
### Changes
- Upgrade hansel to 0.1.78
### Bugs fixed
- Fix printer goto location without thread selection
## 3.7.1 (06-09-2023)
### New Features
### Changes
### Bugs fixed
- Fix ClojureScript double require issue
- Fix java.util.ConcurrentModificationException when building timeline
## 3.7.1 (21-08-2023)
### New Features
- Implement quickjump
- Unblock all breakpoint blocked threads with toolbar and keyboard
### Changes
### Bugs fixed
- Fix enabling/disabling of thread breakpoints
## 3.7.0 (15-08-2023)
### New Features
- Add "Search value on Flows" to taps
- Add "Timeline tool" implementation
- Add power stepping to stepping controls
- Add "Printer tool" implementation
### Changes
### Bugs fixed
- Fix NPE after closing thread tab
## 3.6.10 (19-07-2023)
### New Features
### Changes
- Upgrade to hansel 0.1.74 for a couple of bug fixes. Check hansel changelog.
### Bugs fixed
## 3.6.9 (06-07-2023)
### New Features
### Changes
### Bugs fixed
- Fix ability to capture what happens before the debugger connects on ClojureScript
## 3.6.8 (03-07-2023)
### New Features
- Code stepping follow value allows you to step to the next/prev expression that evaluates to the same value. Useful for understanding how values flow through programs
- Value inspector follow value, same as before.
- New code stepping search tool
### Changes
- The call stack tree search tool was removed since it was buggy and hard to fix because of how javaFx lazy TreeView works
### Bugs fixed
## 3.6.7 (30-06-2023)
### New Features
### Changes
- Upgrade to hansel 0.1.69 with improved coordinate system
### Bugs fixed
- Fix call tree highlight current frame
## 3.6.6 (29-06-2023)
### New Features
- Add debugger window title configurable via :title and flowstorm.title prop
- Add support for debugging litteral maps and sets of any size
### Changes
- Upgrades to hansel 0.1.65 and supports new coordinate system
- Reintroduced #rtrace ^{:thread-trace-limit N}
### Bugs fixed
- issues/65 GUI icon hover making the icon unreadable
## 3.6.5 (16-06-2023)
### New Features
### Changes
- Add flow-storm-find-flow-fn-call to nrepl middleware
### Bugs fixed
## 3.6.4 (13-06-2023)
### New Features
- Add flow-storm.nrepl.middleware for editors integration
### Changes
- Improve step out functionality
### Bugs fixed
## 3.6.3 (06-06-2023)
### New Features
### Changes
- Fix flow-storm.runtime.values/value-type for sorte-maps
- Upgrade hansel to 0.1.63
### Bugs fixed
- Fix step over for the (map some-fn) etc cases
## 3.6.2 (01-06-2023)
### New Features
### Changes
- Exclude org.slf4j/slf4j-nop
### Bugs fixed
## 3.6.1 (30-05-2023)
### New Features
### Changes
- Make pprint panes a TextArea so we can copy it's content
- Upgrade hansel to 0.1.60 to fix ClojureScript namespace instrumentation issue
### Bugs fixed
## 3.6.0 (19-05-2023)
### New Features
- Add functions list refresh button
- Add step-prev-over and step-next-over buttons and keybindings
- Pprint panel now display value type
- Add define all current frame bindings
- Improve auto-scrolling when stepping on the code tool
### Changes
- Keep functions and tree state when switching tabs
- Much improved value inspector
- Update hansel to 0.1.56 so #trace (deftest ...) works
- BREAKING! flow-storm.runtime.values/snapshot-value defmethod was replaced by flow-storm.runtime.values/SnapshotP protocol for performance
### Bugs fixed
- Fix goto-location
- Respect flowstorm.startRecording before :dbg on ClojureStorm
## 3.5.1 (01-05-2023)
### New Features
- Add "Highlight current frame" on the call tree tool
- Add stack tab to code stepping tool
- Add a result pane on the functions list tool
- Add double click on calls tree tool node steps code
- Control recording from the UI
- Add threads breakpoints
- Add basic keyboard support
- Allow DEF button functionality to specify a NS
### Changes
- Made search functionality faster and simplified it's code
- #rtrace automatically opens the code stepping tool in the last position
- Upgrade hansel to 0.1.54
- Signal error when trying to use #rtrace with ClojureStorm
### Bugs fixed
- Don't crash if tracer functions are called before the system is fully started
- Fix Clojure remote debugging race condition
- Keep the thread list split pane size correct after window resize
## 3.4.1 (17-04-2023)
### New Features
### Changes
### Bugs fixed
- [CRITICAL] For remote connections fix repl-watchdog spamming the repl
## 3.4.0 (16-04-2023)
### New Features
- Add support for ClojureStorm
- Add a separate threads list and thread tabs contain names
- Add step up button
- Redesign Flows tools UX
### Changes
- A bunch of performance improvements
### Bugs fixed
- Many bug fixes (sorry for the low detail, this is a big release)
## 3.3.325 (09-04-2023)
### New Features
### Changes
- Exclude guava as a transitive dep in tools.build since it breaks shadow-cljs 2.21.0
- Upgrade hansel to 0.1.50
### Bugs fixed
## 3.3.320 (16-02-2023)
### New Features
### Changes
- Upgrade openjfx to 19.0.2.1
- Upgrade hansel to 0.1.46
### Bugs fixed
## 3.3.315 (30-01-2023)
### New Features
### Changes
- Upgrade to hansel 0.1.42 which contains a couple of bug fixes
### Bugs fixed
## 3.3.313 (29-01-2023)
### New Features
### Changes
### Bugs fixed
- Update to hansel 0.1.38 which contains a couple of bug fiexs
## 3.3.309 (29-12-2022)
### New Features
### Changes
- Improve docs file generation. Generated docs will be put into flow-docs.edn instead of samples.edn since
it is less likely to collide. Also the data format has been improved for extensibility
### Bugs fixed
## 3.3.307 (26-12-2022)
### New Features
### Changes
- Update hansel dependency to 0.1.35
### Bugs fixed
## 3.3.303 (20-12-2022)
### New Features
### Changes
### Bugs fixed
- Handle OS theme detector exceptions
## 3.3.301 (16-12-2022)
### New Features
### Changes
- Update hansel to 0.1.31 which contains a critical bug
- Improve value inspector styles
### Bugs fixed
- Fix docs examples display
### 3.3.295 (14-12-2022)
## New Features
- Add Flow Docs - Generate projects functions documentation by sampling their executions.
### Changes
### Bugs fixed
- Fix a ConcurrentModificationException on debuggers_api/reference_frame_data!
## 3.2.283 (29-11-2022)
## New Features
- Add browser var recursive instrumentation
### Changes
### Bugs fixed
- Update to hansel 0.1.22 to fix ClojureScript go blocks instrumentation
## 3.2.271 (16-11-2022)
## New Features
- New tap button allows you to tap> any value from the UI (nice to integrate with other tooling like portal)
- New locals "tap value" allows you to tap> any locals
### Changes
- Migrate to hansel for instrumentation (should be much better than previous instrumentation system)
- Value inspector navigation bar now shows keys instead of val text
### Bugs fixed
- Fix automatic [un]instrumentation watcher
## 3.1.263 (18-10-2022)
## New Features
### Changes
### Bugs fixed
- Fix clojure instrument entire namespace for non libraries
## 3.1.261 (18-10-2022)
## New Features
### Changes
- Remove flow-storm hard dependency on org.clojure/clojurescript artifact. Will lazy require when needed for ClojureScript, assuming the dependency will be provided
### Bugs fixed
## 3.1.259 (17-10-2022)
## New Features
- Add #rtrace ^{:thread-trace-limit X} where X can be a integer. Execution will throw after tracing X times for a trace. Useful for debugging possibly infinite loops
- Add support for snapshoting mutable values via flow-storm.runtime.values/snapshot-value multimethod
- Add #tap-stack-trace, to tap the current stack trace wherever you add it
- Add support for core.async/go blocks instrumentation
- Add Ctrl+MouseWheel on forms to step prev/next
### Changes
- Immediately highlihgt the first trace when creating a flow in the debugger
- Remove unnecessary first-fn-call-event
### Bugs fixed
- Alt+Tab now works on MacOs for switching between the debugger and the repl windows (thanks to Lucy Wang @lucywang000)
- Fix extend-protocol and extend-type instrumentations for ClojureScript
- Fix instrumentation breaking variadic functions in ClojureScript
## 3.0.236 (7-10-2022)
## New Features
### Changes
- Improves theming
- Fix dynamic vars not being re-evaluated as dynamic in cljs
### Bugs fixed
- Fix dynamic vars not being re-evalueated as dynamic in ClojureScript
- Fix timeout issues on remote connection
## 3.0.231 (6-10-2022)
## New Features
- Add `Instrument form without bindings` to form context menu
- Add got to last trace on code loops context-menu
### Changes
- Full [un]instrumentation synchronization between the browser and #trace (Clojure only)
- Now you can [un]instrument single vars from the browser, even if they where defined at the repl (Clojure only)
- Improved Single var [un]instrumentation from the browser (Clojure and ClojureScript)
### Bugs fixed
- Fix show-error on the clojure local path
## 3.0.216 (29-9-2022)
## New Features
- flow-storm.api/cli-run now supports :flow-id key
### Changes
### Bugs fixed
- Windows pprint form problem
- Bunch of minor small bug fixes
## 3.0.208 (28-9-2022)
## New Features
### Changes
- Instrummented code should run much faster due to removed unnecessary runtime ctx rebinding
### Bugs fixed
- Fix browser navigation after instrumentation bug
## 3.0.198 (24-9-2022)
## New Features
- Automatic event retention. For ClojureScript or remote Clojure you don't need to remote-connect by hand
when you want to capture traces before the debugger is connected. Just need to require flow-storm.api on your main.
- Automatic ui cleaning on reconnection
### Changes
- Remote connection now accepts :debugger-host and :runtime-host keys (check out the documentation)
### Bugs fixed
- Fix the more button on value inspector
- Fix for infinite sequence handling
## 3.0.188 (22-9-2022)
## New Features
- Automatic remote connection management. The re-connect button was removed from the toolbar since it isn't needed anymore
### Changes
### Bugs fixed
- Fix javafx platform not initialized exception when there is a error connecting to a repl
## 3.0.184 (21-9-2022)
## New Features
### Changes
### Bugs fixed
- Add support for remote debugging without repl connection (clojure and clojurescript)
- Show nrepl errors on the UI
- Fix ClojureScript re-run flow
- Fix a deadlock caused by the event system
## 3.0.173 (15-9-2022)
## New Features
3.0 is a full redesign! So it is full of changes, and fixes. Most remarkable things are :
- ClojureScript is feature par with Clojure now, so every feature is available to both languages.
- Remote debugging can be accomplished by just connecting to nrepl server (socket repl support on the roadmap)
- A programable API (https://jpmonettas.github.io/flow-storm-debugger/user_guide.html#_programmable_debugging)
- Enables the posibility to integrate it with IDEs/editors
### Changes
### Bugs fixed
## 2.3.141 (15-08-2022)
## New Features
* Add inspect for locals
* Can jump to any index by typing the idx number in the thread controls
### Changes
* Locals print-length is now 20 and print-level 5
* Make the value inspector show more info on dig nodes
### Bugs fixed
## 2.3.131 (10-08-2022)
## New Features
* Add a proper lazy and recursive value inspector
* Add tap tool (support for tap>)
* New functions for shutting down the debugger and connections gracefully.
When starting with `flow-storm.api/local-connect` or `flow-storm.api/remote-connect` you can shut it down with `flow-storm.api/stop`
When starting a standalone debugger with `flow-storm.debugger.main/start-debugger` you can shutdown with `flow-storm.debugger.main/stop-debugger`
* Add support for light and dark themes, in selected or automatic mode. Checkout the user guide for more info. (thanks to Liverm0r!)
* Thread tabs can be closed and reordered
### Changes
* The entire debugger was refactored to manage state with mount
### Bugs fixed
* Fix #28 - Callstack tree args and ret listviews should expand to the bottom
* Fix #34 - instrument-forms-for-namespaces not instrumenting (def foo (fn [...] ...))
## 2.2.114 (07-07-2022)
## New Features
* Add bottom bar progress indicator when running commands
* Add reload tree button on callstack tree tabs
* Add search bar to functions list
* This release also contains big internal refactors to make the codebase cleaner and more efficient
### Changes
* Automatically deref all reference values in traces
* Automatically change to flow tab on new flow
### Bugs fixed
## 2.2.99 (10-06-2022)
## New Features
* Add jump to first and last traces on thread controls (useful for exceptions debugging)
* Add print-level and print-meta controls on pprint value panels
* Improve re-run flow UX
* Namespace instrumentation now accepts :verbose? to log known and unknown instrumentation errors details
* Add flow-storm.api/uninstrument-forms-for-namespaces to undo instrument-form-for-namespaces instrumentation
* Add ctx menu on locals to define vars from values
* Add browser namespaces instrumentation/uninstrumentation
* Add browser instrumentation synchronization (for everything but #trace)
* Add #rtrace0 ... #rtrace5, like #rtrace but with different flow-ids
* Add double clicking on flows functions window executes show function calls
### Changes
* Remove flow-storm.api/run since #rtrace(runi) is enough
* Flows functions window now have checkboxes for selecting fncall arguments to print
### Bugs fixed
* Fix re run flow for #rtrace case
* Fix local binding instrumentation and debugging
## 2.2.68 (10-06-2022)
## New Features
* Add def value button on every value panel to define the value so you can work with it at the repl
* Add namespaces browser with instrumentation capabilities
### Changes
### Bugs fixed
## 2.2.64 (09-06-2022)
## New Features
* Add conditional tracing via #ctrace and ^{:trace/when ...} meta
### Changes
### Bugs fixed
## 2.2.59 (06-06-2022)
## New Features
### Changes
### Bugs fixed
* Fix run-command for the local connection path
## 2.2.57 (06-06-2022)
## New Features
* Add Clojurescript support
* Remote debugging via `flow-storm.api/remote-connect` and `flow-storm.api/cli-run`
### Changes
* `flow-storm.api/cli-run` now accepts :host and :port
### Bugs fixed
## 2.0.38 (02-05-2022)
## New Features
* Add styles customization via a user provided styles file
* The debugger can instrument and debug itself
### Changes
### Bugs fixed
## 2.0.0 (18-04-2022)
================================================
FILE: Makefile
================================================
.PHONY: clean docs test lint-dbg lint-inst install-dbg install-inst deploy-dbg deploy-inst
docs: docs/user_guide.adoc
asciidoctorj -b html5 -o docs/user_guide.html docs/user_guide.adoc
clean:
clj -T:build clean
test:
clj -M:test:dev unit-clj
lint:
clj-kondo --config .clj-kondo/config.edn --lint src-dbg src-shared src-inst
flow-storm-dbg.jar:
clj -T:build jar-dbg
flow-storm-inst.jar:
clj -T:build jar-inst
install-dbg: flow-storm-dbg.jar
mvn install:install-file -Dfile=target/flow-storm-dbg.jar -DpomFile=target/classes/META-INF/maven/com.github.flow-storm/flow-storm-dbg/pom.xml
install-inst: flow-storm-inst.jar
mvn install:install-file -Dfile=target/flow-storm-inst.jar -DpomFile=target/classes/META-INF/maven/com.github.flow-storm/flow-storm-inst/pom.xml
deploy-dbg:
mvn deploy:deploy-file -Dfile=target/flow-storm-dbg.jar -DrepositoryId=clojars -DpomFile=target/classes/META-INF/maven/com.github.flow-storm/flow-storm-dbg/pom.xml -Durl=https://clojars.org/repo
deploy-inst:
mvn deploy:deploy-file -Dfile=target/flow-storm-inst.jar -DrepositoryId=clojars -DpomFile=target/classes/META-INF/maven/com.github.flow-storm/flow-storm-inst/pom.xml -Durl=https://clojars.org/repo
================================================
FILE: Readme.md
================================================
# Flow-storm debugger
This is the central repository for [FlowStorm](http://www.flow-storm.org/) an omniscient time travel debugger for Clojure and ClojureScript.
There are two ways of using it :
- [With ClojureStorm](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_clojurestorm) (recommended) : For dev, swap your Clojure compiler by ClojureStorm and get everything instrumented automatically
- [Vanilla FlowStorm](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_vanilla_flowstorm) : Just add FlowStorm to your dev classpath and instrument by re-evaluating forms
ClojureStorm is a fork of the official Clojure compiler that adds automatic instrumentation so you don't need to think about it (you can still disable it when you don't need it).
You use it by swapping the official Clojure compiler by ClojureStorm at dev time, using dev aliases or profiles.
# Artifacts
FlowStorm latest stable releases :
- The complete debugger (includes `flow-storm-inst`)
- `[com.github.flow-storm/flow-storm-dbg "4.5.9"]`
- A slimmer version with no GUI, to use it for Clojure or ClojureScript remote debugging
- `[com.github.flow-storm/flow-storm-inst "4.5.9"]`
ClojureStorm latest stable releases :
- Clojure 1.12
- `[com.github.flow-storm/clojure "1.12.4"]`
- Clojure 1.11
- `[com.github.flow-storm/clojure "1.11.4-10"]`
ClojureScriptStorm latest stable releases :
- ClojureScript 1.12.116
- `[com.github.flow-storm/clojurescript "1.12.134-3"]`
- ClojureScript 1.11.132
- `[com.github.flow-storm/clojurescript "1.11.132-9"]`
# Prerequisites
- jdk17+ (if you still need to run it with jdk11 take a look at [here](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_run_with_jdk_11))
- clojure 1.11.0+
- clojure 1.10.* only supported if you use it from source, like `{:git/url "https://github.com/flow-storm/flow-storm-debugger" :git/sha "..."}`
# QuickStart and Documentation
If you want to use it with Clojure checkout the [Clojure QuickStart guide](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_clojure)
or the [ClojureScript QuickStart Guide](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_clojurescript) if you are using ClojureScript.
Please refer to the [user's guide](https://flow-storm.github.io/flow-storm-debugger/user_guide.html) for a list of features and how to use them.
# ClojureStorm and ClojureScriptStorm
*ClojureStorm* is a dev compiler. It is a fork of the official Clojure compiler enhanced with automatic instrumentation.
To use it, you just swap it with your normal Clojure compiler at dev time (by using deps cli aliases or lein profiles) to improve
your development experience, while making sure you use your normal compiler for everything else (tests and production).
*ClojureScriptStorm* is the same as ClojureStorm but for ClojureScript, so a fork of the official ClojureScript compiler is enhanced with automatic instrumentation.
ClojureStorm sources are here : https://github.com/flow-storm/clojure
ClojureScriptStorm sources are here : https://github.com/flow-storm/clojurescript
# Features
Flow storm debugger is packed with a ton of features, which the [user's guide](https://flow-storm.github.io/flow-storm-debugger/user_guide.html)
covers in detail.
## Information for developers
If you want to enhance, fix, debug, or just learn about the internals of FlowStorm take a look at
[here](./docs/dev_notes.md)
## Some demo videos (newers at the top)
- [FlowStorm demo at Clojure Apropos](https://www.youtube.com/watch?v=a-PrBjlBdw8)
- [ClojureScript compiler fun with FlowStorm](https://www.youtube.com/watch?v=YYHRx3EnPmg)
- [Don't fear the storm](https://www.youtube.com/watch?v=CspQX_R0NbM)
- [Clojure visual-tools 29 - FlowStorm 4.1 workflows](https://www.youtube.com/watch?v=9nY25hwzWRc)
- [Clojure web apps with FlowStorm 3.17](https://www.youtube.com/watch?v=h8AFpZkAwPo)
- [Reifying execution, the interactive programming missing piece](https://www.youtube.com/watch?v=BuSpMvVU7j4&t=1394s)
- [FlowStorm printer demo](https://www.youtube.com/watch?v=06-MA4HSS24)
- [Smashing a real ClojureScript bug with FlowStorm](https://www.youtube.com/watch?v=4VXT-RHHuvI)
- [Debugging Clojure with FlowStorm 3.6](https://www.youtube.com/watch?v=Mmr1nO6uMzc)
- [Searching and following values](https://www.youtube.com/watch?v=CwXhy-QsZHw)
- [Show me your REPL episode](https://www.youtube.com/watch?v=2nH59edD5Uo)
- [Debugging Clojure with FlowStorm](https://www.youtube.com/watch?v=PbGVTVs1yiU)
- [Debugging ClojureScript with FlowStorm](https://www.youtube.com/watch?v=jMYl32lnMhI)
- [Presentation at London Clojurians](https://www.youtube.com/watch?v=A3AzlqNwUXc)
- [Flows basics](https://www.youtube.com/watch?v=YnpQMrkj4v8)
- [Instrumenting libraries](https://youtu.be/YnpQMrkj4v8?t=332)
- [Debugging the ClojureScript compiler](https://youtu.be/YnpQMrkj4v8?t=533)
- [Browser](https://www.youtube.com/watch?v=cnLwRzxrKDk)
- [Def button](https://youtu.be/cnLwRzxrKDk?t=103)
## FAQ
### Clojure has the repl, why does it need a debugger?
In [this talk](https://www.youtube.com/watch?v=A3AzlqNwUXc&t=934s) I tried to argue that even as amazing as it is to have a repl to poke around,
there are some inconveniences that I think can be greatly improved by a debugger.
- Defining function parameters and locals with def (for sub form execution) isn't easy for complex values
- When functions contains loops, maps, filters, etc with anonymous functions is hard to capture every value for further inspection.
- Being Clojure a dynamic lang, running the program in my head isn't easy if I'm not familiar with the code base
- Adding prints (or tap>) is inconvenient because you need to guess where the problem probably is first.
- Debugging statements needs to be constantly typed and removed which gets repetitive and annoying
- Exploring complex values at the console is tedious, that is why tools like portal, reveal, rebl, etc exist.
Some of the issues there can be alleviated by adding libraries like scope capture, and portal but it isn't straight forward to integrate them
and even if it was IMHO there is a lot to be gained from a proper integrated debugging system.
So I want to stop guessing and a tool that allows me to see what is happening when a Clojure program runs, be it a small expression or an entire codebase.
I want it to be useful for cases when I'm chasing a bug or when I just want to understand how something works.
I also think some Clojure constraints, like immutability and being expression based, allows us to go beyond repl poking and steppers.
We can record everything that happens when a program runs and then inspect the execution using multiple tools (being a stepper one of them) and fully
integrating it with the repl which also enhance the repl poking experience.
### What's that magic? How does it work?
The idea behind FlowStorm is pretty simple :
- Instrument Clojure code
- Run it
- Record everything that happens on each thread during execution into timelines
- Provide a GUI with multiple tools to explore the recorded values and execution flows
The interesting part here I guess are instrumentation and recording.
FlowStorm can instrument your code in two ways :
- The ClojureStorm way (recommended for Clojure) which swaps your official Clojure compiler with a patched one (only for dev) that emits extra JVM bytecode everytime you compile something
to record everything that is happening. This method provides automatic instrumentation everywhere, which is very practical. You still get to un-instrument things if you need to do
things like measure performance, which isn't going to be accurate with the extra bytecode.
- The vanilla way (can be used for Clojure and is the only option for ClojureScript), that just grabs the Clojure source expressions you are interested in, walks the AST, instruments it, and re-evals the
instrumented version through the repl. You do this by using reader tags (like `#trace (defn foo [...] ...)`) or by using the FlowStorm browser tab to instrument entire namespaces.
Doesn't matter which method instrumented the code, when it runs it will record every expression in a timeline.
Because Clojure is expression based and most data is immutable, recording is just retaining JVM references together with the source coordinate of each expression. This is pretty fast and a simple way of recording execution.
The timeline is a structure that provides efficient insertion, and efficient access sequentially and as a functions call tree.
You can see some diagrams here : https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_internals_diagrams_and_documentation
### Isn't recording everything too expensive? Like, does it work with something like a game loop?
The answer here is it depends. First, not everything needs to be recorded all the time. FlowStorm provides easy controls for start/stop recording, instrumenting/un-instrumenting stuff,
freeing recordings, and for keeping an eye on the heap space which you can also control via JVM parameters.
For a lot of applications it is probably fine even if you keep recording everything. For applications that generate a lot of recordings you can keep the recording off and just enable it
with one click right before executing the action you want to record, and then turn it off again.
For things like game loops you can also use the thread breakpoints functionality which provides a way of pausing/resuming threads at specific points so you have more control on how they
execute. Combining that with start/stopping recording is probably enough to debug this kind of application.
### Aren't omniscient debuggers slow?
Omniscient debuggers have a reputation of being slow, and I think a bunch of them are. IMHO a lot of the slowness comes from how things need to be recorded and replayed on mutable languages.
For Clojure, just retaining references is enough, and looking into a value at any point in time is just looking at it. Since most references point to immutable values, they don't need to be
reconstructed by applying state changes for each step.
There are mutable language omniscient debuggers that implement stepping backwards fast enough to be useful, but trying to implement things like searching or following values is probably going to be
much slower since everything needs to be reconstructed at each step.
### How does it compare to other Clojure debuggers?
There are many comparison points with different Clojure debugging tools, I'll try to list some of them here.
If you feel something here is unfair or needs to be clarified please open an issue or submit a PR. Also if you would like to see here how it compares with any other debugging tool let me know.
The first big difference is that FlowStorm works for Clojure and ClojureScript while most debuggers like Cider/VSCode/Cursive only work for Clojure. So once you learn
to use FlowStorm you can use it with both languages.
The second imho big difference is that most debugging tools are designed to understand small pieces of execution you could be interested in, while FlowStorm
is designed to look at a program execution as a whole, to be used not only while chasing bugs but also as a development companion to help you reason about whatever
system you are running.
Most tools like Cider, Cursive or VScode debuggers are blocking steppers and require you to add breakpoints to specific places, which is okay if you have an idea
already of where the bug could be located, but they fall short if you aren't sure, or you just want to use the tool to understand an entire codebase execution.
#### FlowStorm VS Cursive (or Browser's for ClojureScript) debuggers
I think the main difference here is FlowStorm being expression and value oriented while the Cursive and Browser's one being line and memory poking oriented,
which I don't think is very useful in Clojure[Script]. If you want to experience what I mean, try to debug this exception using Cursive vs FlowStorm/Cider/VSCode :
```
(defn foo [n]
(->> (range n)
(filter odd?)
(partition-all 2)
(map second)
(drop 10)
(reduce +)))
(foo 70)
```
On the other side, the nice thing about the Cursive debugger is that you can use it to step over Java code which is useful if you have a mixed lang codebase. But for those cases
you can always use both.
#### FlowStorm VS Cider/VSCode debuggers
Cider and VSCode debuggers are steppers, so I'll only be comparing it against the stepping and value inspection capabilities of FlowStorm, but keep in mind that FlowStorm provides
many more features than stepping. Cider provide some tracing capabilities to trace entire namespaces but it relies on printing to the console which I haven't found useful outside of very specific situations, so
I don't think it is useful to understand an entire codebase.
For the steppers, the main difference is that Cider/VSCode are blocking debuggers, which are nice in some situations but only provide stepping forward, while FlowStorm allows you to
step in many different ways.
Another difference is that currently on Cider/VSCode you can't step into functions unless you have previously instrumented them, which for the most isn't a problem in FlowStorm if you are using ClojureStorm.
A nice thing about Cider/VSCode steppers is that you get to step over your editor source code, which can also be done in FlowStorm if you use an integration like CiderStorm.
There is also some code that currently can't be stepped with Cider/VSCode that can be with FlowStorm. This is code that happens to be inside literal sets of maps with more than 8 keys.
If you want to compare it try stepping code like this in both debuggers :
```
(defn bla []
{(+ 1 2) (first #{3})
2 (+ 1 1)
4 (+ 2 2)
6 (+ 3 3)
8 (+ 4 4)
10 (+ 5 5)
12 (+ 6 6)
14 (+ 7 7)
16 (+ 8 8)})
(bla)
```
## What to do when things don't work?
Please create a [issue](https://github.com/flow-storm/flow-storm-debugger/issues) if you think you found a bug.
If you are not sure you can ask in :
- [#flow-storm slack channel](https://clojurians.slack.com/archives/C03KZ3XT0CF)
- [github discussions](https://github.com/flow-storm/flow-storm-debugger/discussions)
## Acknowledgements
Big thanks to [Roam Research](https://roamresearch.com/), [Nubank](https://nubank.com.br/) and all previous and current sponsors.
Thanks to [Cider](https://github.com/clojure-emacs/cider/) debugger for inspiration and some clever ideas for code instrumentation.
================================================
FILE: UNLICENSE
================================================
This is free and unencumbered software released into the public domain.
Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.
In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
For more information, please refer to
================================================
FILE: build.clj
================================================
(ns build
(:require [clojure.tools.build.api :as b]
[clojure.string :as str]
[clojure.java.io :as io]
[clojure.spec.alpha :as s]))
(def version (or (System/getenv "VERSION")
"4.5.9"))
(def target-dir "target")
(def class-dir (str target-dir "/classes"))
(defn clean [_]
(b/delete {:path target-dir}))
(def aot-compile-nses
"Precompile this so clojure storm can call flow-storm.storm-api functions
at repl init and it doesn't slow down repl startup."
['flow-storm.storm-api
'flow-storm.utils
'flow-storm.api
'flow-storm.runtime.debuggers-api
'flow-storm.runtime.types.fn-call-trace
'flow-storm.runtime.indexes.timeline-index
'flow-storm.runtime.indexes.api
'flow-storm.runtime.indexes.protocols
'flow-storm.runtime.types.fn-return-trace
'flow-storm.runtime.events
'flow-storm.runtime.indexes.thread-registry
'flow-storm.runtime.indexes.form-registry
'flow-storm.runtime.values
'flow-storm.runtime.types.bind-trace
'flow-storm.runtime.types.expr-trace
'flow-storm.runtime.indexes.utils])
(def aot? (if-let [aot (System/getenv "AOT")]
(read-string aot)
;; if no AOT value provided we default to true
true))
(defn- check-jvm []
(let [jvm-version (-> (System/getProperty "java.specification.version")
Integer/parseInt)]
(when (>= jvm-version 21)
(throw (ex-info "Not building with JVM >= 21 because of the SequencedCollection issue. See https://aphyr.com/posts/369-classnotfoundexception-java-util-sequencedcollection" {})))
(println "Building with JVM " jvm-version)))
(defn jar-dbg [_]
(check-jvm)
(clean nil)
(println "AOT compiling dbg : " aot?)
(let [lib 'com.github.flow-storm/flow-storm-dbg
basis (b/create-basis {:project "deps.edn"})
jar-file (format "%s/%s.jar" target-dir (name lib))
src-dirs ["src-dbg" "src-shared" "src-inst"]]
(b/write-pom {:class-dir class-dir
:lib lib
:version version
:basis basis
:src-dirs src-dirs
:pom-data [[:licenses
[:license
[:name "Unlicense"]
[:url "http://unlicense.org/"]]]]})
(when aot?
(b/compile-clj {:basis basis
:src-dirs src-dirs
:class-dir class-dir
:compile-opts {:direct-linking false}
:ns-compile aot-compile-nses}))
(b/copy-dir {:src-dirs (into src-dirs ["resources"])
:target-dir class-dir})
(b/jar {:class-dir class-dir
:jar-file jar-file})))
(defn jar-inst [_]
(check-jvm)
(clean nil)
(println "AOT compiling inst : " aot?)
(let [lib 'com.github.flow-storm/flow-storm-inst
src-dirs ["src-inst" "src-shared"]
basis (b/create-basis {:project nil
:extra {:deps {'org.java-websocket/Java-WebSocket {:mvn/version "1.5.3"}
'com.cognitect/transit-clj {:mvn/version "1.0.333"}
'com.cognitect/transit-cljs {:mvn/version "0.8.280"}
'com.github.flow-storm/hansel {:mvn/version "0.1.90"}
'org.clojure/data.int-map {:mvn/version "1.2.1"}
'amalloy/ring-buffer {:mvn/version "1.3.1"}}
:paths src-dirs}})
jar-file (format "%s/%s.jar" target-dir (name lib))]
(b/write-pom {:class-dir class-dir
:lib lib
:version version
:basis basis
:src-dirs src-dirs
:pom-data [[:licenses
[:license
[:name "Unlicense"]
[:url "http://unlicense.org/"]]]]})
(when aot?
(b/compile-clj {:basis basis
:src-dirs src-dirs
:class-dir class-dir
:compile-opts {:direct-linking false}
:ns-compile aot-compile-nses}))
(b/copy-dir {:src-dirs src-dirs
:target-dir class-dir})
(b/jar {:class-dir class-dir
:jar-file jar-file})))
================================================
FILE: deps.edn
================================================
{:paths ["src-inst" "src-dbg" "src-shared" "resources"]
:deps {;; IMPORTANT !!
;; If adding any dependency for the `inst` part also add it on
;; build.clj jar-inst
org.java-websocket/Java-WebSocket {:mvn/version "1.5.3"}
com.cognitect/transit-clj {:mvn/version "1.0.333"}
com.cognitect/transit-cljs {:mvn/version "0.8.280"}
com.github.flow-storm/hansel {:mvn/version "0.1.90"}
org.openjfx/javafx-controls {:mvn/version "21.0.4-ea+1"}
org.openjfx/javafx-base {:mvn/version "21.0.4-ea+1"}
org.openjfx/javafx-graphics {:mvn/version "21.0.4-ea+1"}
org.openjfx/javafx-web {:mvn/version "21.0.4-ea+1"}
org.kordamp.ikonli/ikonli-javafx {:mvn/version "12.3.1"}
org.kordamp.ikonli/ikonli-materialdesign-pack {:mvn/version "12.3.1"}
com.github.jpmonettas/j-system-theme-detector {:mvn/version "3.8.1"}
nrepl/nrepl {:mvn/version "1.1.1"}
org.clojure/data.int-map {:mvn/version "1.2.1"}
org.fxmisc.richtext/richtextfx {:mvn/version "0.11.1"}
amalloy/ring-buffer {:mvn/version "1.3.1"}}
:aliases {:cljs-storm {:classpath-overrides {org.clojure/clojurescript nil} ;; disable the official compiler
:extra-deps {thheller/shadow-cljs {:mvn/version "2.27.4"
:exclusions [org.clojure/clojurescript]}
;; bring ClojureScriptStorm
com.github.flow-storm/clojurescript {:mvn/version "1.11.132-2"}
;; add FlowStorm runtime dep
com.github.flow-storm/flow-storm-inst {:local/root "." #_#_:mvn/version "RELEASE"}
cider/cider-nrepl {:mvn/version "0.28.3"}
refactor-nrepl/refactor-nrepl {:mvn/version "3.5.2"}
cider/piggieback {:mvn/version "0.5.2"}}
:jvm-opts ["-Dcljs.storm.instrumentOnlyPrefixes=dev"
"-Dcljs.storm.instrumentEnable=true"]}
:storm {:classpath-overrides {org.clojure/clojure nil}
:extra-deps {com.github.flow-storm/clojure {:mvn/version "1.12.4"}}
:jvm-opts ["-Dflowstorm.theme=dark"
"-Dclojure.storm.instrumentAutoPrefixes=false"
"-Dclojure.storm.instrumentOnlyPrefixes=dev-tester"
"-Dflowstorm.jarEditorCommand=emacsclient -n +<>:0 <>/<>"
"-Dflowstorm.fileEditorCommand=emacsclient -n +<>:0 <>"]}
:fs-timelines-counters-plugin {:extra-deps {timelines-counters/timelines-counter {:local/root "./examples/plugins/basic-plugin/"}}
:jvm-opts ["-Dflowstorm.plugins.namespaces.timelines-counters=flow-storm.plugins.timelines-counters.all"]}
:dev {:extra-paths ["src-dev" "classes"]
:extra-deps {
org.openjfx/javafx-swing {:mvn/version "21.0.4-ea+1"} ;; for scenic view to run
io.github.tonsky/clj-reload {:mvn/version "0.7.1"}
}
:jvm-opts ["-Dvisualvm.display.name=FlowStorm"
;; for the profilers
"-Djdk.attach.allowAttachSelf" "-XX:+UnlockDiagnosticVMOptions" "-XX:+DebugNonSafepoints"]}
:build {:extra-deps {io.github.clojure/tools.build {:git/tag "v0.9.6" :git/sha "8e78bcc"}}
:ns-default build
:jvm-opts ["-Dcljfx.skip-javafx-initialization=true"] }
:test {:extra-paths ["test"]
:extra-deps {lambdaisland/kaocha {:mvn/version "1.70.1086"}
org.clojure/clojurescript {:mvn/version "1.11.60"}}
:jvm-opts ["-Xmx10500m"]
:main-opts ["-m" "kaocha.runner"]}}}
================================================
FILE: docs/_config.yml
================================================
theme: jekyll-theme-cayman
================================================
FILE: docs/dev_notes.md
================================================
# Developer notes
The purpose of this document is to collect information and tips for people wanting to enhance, fix, debug,
or just learn about the internals of FlowStorm.
## FlowStorm design
FlowStorm is made of three parts :
* An __instrumentation__ system
* A __runtime__ system
* A __debugger__ system
If you want some diagrams of how all this works together take a look at
[here](https://raw.githubusercontent.com/flow-storm/flow-storm-debugger/master/docs/high_level_diagram.svg)
### Instrumentation
The __instrumentation__ system is responsible for instrumenting your code. This means interleaving extra code to trace
what your program is doing. There are currently 3 ways of instrumenting :
* [Hansel](https://github.com/flow-storm/hansel) a library to add instrumentation by re-writing forms at macroexpansion time
* [ClojureStorm](https://github.com/flow-storm/clojure) a Clojure dev compiler that will instrument by emitting extra bytecode
* [ClojureScriptStorm](https://github.com/flow-storm/clojurescript) a ClojureScript dev compiler that will instrument by emitting extra javascript
They are all independent from FlowStorm but you need to choose one of them to use FlowStorm with your programs.
When starting, FlowStorm will setup callbacks to them, which they will use when generating instrumentation.
You can see how it hooks with each of them in :
`flow-storm.tracer/[hansel-config | hook-clojure-storm | hook-clojurescript-storm]`
Whatever instrumentation system the user chooses, when instrumented code runs it will hit :
* `flow-storm.tracer/trace-fn-call`
* `flow-storm.tracer/trace-fn-return`
* `flow-storm.tracer/trace-fn-unwind`
* `flow-storm.tracer/trace-expr-exec`
* `flow-storm.tracer/trace-bind`
See the __runtime__ for what happens next.
### Runtime
The __runtime__ is FlowStorm's subsystem that runs inside the debuggee process.
Its responsibility is to record all the traces that arrive via the `flow-storm.tracer/trace-*` set of functions while
`flow-storm.tracer/recording` is `true`.
It will record everything in two registries `flow-storm.runtime.indexes.api/forms-registry` and
`flow-storm.runtime.indexes.api/flow-thread-registry`. Take a look at their docs strings for more info.
The forms registry will store all the instrumented forms by form-id, which is a hash of the form.
The threads registry on the other side will be storing one timeline per thread plus a multi-thread timeline when
its recording is on.
The timeline is the main recording structure, and what every FlowStorm functionality is build upon.
You can see a diagram of the timeline internal structure
[here](https://raw.githubusercontent.com/flow-storm/flow-storm-debugger/master/docs/timeline.svg).
It is currently implemented as `flow-storm.runtime.indexes.timeline-index/ExecutionTimelineTree` type, which internally
uses a mutable list. It would be simpler if this could be an immutable list but the decision was made because it needs
to be fast to build, and without too much garbage generation, so we don't make the debuggee threads too slow.
With the current architecture transients can't be used because there isn't a trace that indicates that a thread is done,
so it can't be persisted. Maybe it can be done in the future if some kind of batching by time is implemented.
All objects that represent traces are defined by types in `flow-storm.runtime.types.*` instead of maps. This is to
reduce memory footprint.
All the timelines can be explored from the repl by using the functions defined in `flow-storm.runtime.indexes.api`.
The __runtime__ exposes all the indexes functionality to debuggers via `flow-storm.runtime.debuggers-api`. The main
difference between this and `flow-storm.runtime.indexes.api` is that the former will return value ids instead of actual
values, since not all values can leave the debuggee process (think of infinite sequences), and also because of
performance, since most of the time, for big nested values, the user is interested in a small part of them and
serializing is expensive.
When debugging locally, the functions in `flow-storm.runtime.debuggers-api` will be called directly, while they will be
called through a websocket in the remote case.
The __runtime__ part is packaged into `com.github.flow-storm/flow-storm-inst` as well as in
`com.github.flow-storm/flow-storm-dbg` artifacts.
### Debugger
The __debugger__ is the part of FlowStorm that implements all the tools to explore the recordings with a GUI.
Its main entry point is `flow-storm.debugger.main/start-debugger`.
It has a bunch of subsystems that implement different aspects of it :
* `flow-storm.debugger.state/state`
* `flow-storm.debugger.runtime-api/rt-api`
* `flow-storm.debugger.ui.main/ui`
* `flow-storm.debugger.events-queue/events-queue`
* `flow-storm.debugger.websocket/websocket-server`
* `flow-storm.debugger.repl.core/repl`
All namespaces have doc strings explaining they purpose and how they work.
Not all subsystems will be running in all modes. For example the `websocket-server` and the `repl` client will be only
running in remote debugging mode, since they are not needed for local debugging.
The __debugger__ uses a custom component system defined in `flow-storm.state-management`, which is a very simple `mount`
like component system. It doesn't use `mount` since we don't want to drag too many dependencies to the user classpath.
The __debugger__ GUI is implemented as a JavaFX application, with all screens implemented in `flow-storm.debugger.ui.*`.
### The coordinate system
All expressions traces will contain a `:coord` field, which specifies the coordinate inside the form with id `:form-id`
a value refers to.
The coordinates are stored as a string, like `"3,2"`. They are stored as strings for performance reasons. This is
because on ClojureStorm, they can be compiled to the strings constant pool, which will be interned when the class is loaded and
they can thereof be referenced by traces.
As an example, the coordinate `"3,2"` for the form :
```clojure
(defn foo [a b] (+ a b))
```
refers to the second symbol `b` in the `(+ a b)` expression, which is under coordinate `3`.
The coordinates are a zero based collection of positional indexes from the root, for all but for maps
and sets.
For them instead of an index it will be, for a :
map key : a string K followed by the hash of the key form
map value: a string V followed by the hash of the key form for the val
For sets it will also be a string K followed by the hash of the set
element form.
As an example :
(defn foo [a b]
(assoc {1 10
(+ 42 43) 100}
:x #{(+ 1 2) (+ 3 4) (+ 4 5) (+ 1 1 (* 2 2))}))
some examples coordinates :
[3 1 "K-240379483"] => (+ 42 43)
[3 2 "K1305480196" 3] => (* 2 2)
You can see an implementation of this hash function in `hansel.utils/clojure-form-source-hash`.
The reason we can't use indices for maps and sets is because the order is not guaranteed.
### Values
As described in the __runtime__ section, recorded values never leave `flow-storm.runtime.debuggers-api`.
They are stored into a registry, and a reference to them is returned. This references are defined in
`flow-storm.types/ValueRef`, which acts as a reified pointer.
Whatever is using `flow-storm.runtime.debuggers-api` will deal with this `ValueRef`s instead of actual values.
Functions are also exposed by `flow-storm.runtime.debuggers-api` to work with `ValueRef`s :
* `val-pprint` for printing a value into a string representation with provided `print-level` and `print-length`
The values registry implementation is a little more involved than just a map from (hash value) -> value because not all
values can be hashed, specially infinite sequences. For that, every value is wrapped in a
`flow-storm.runtime.values/HashableObjWrap` which will use `flow-storm.utils/object-id` for the hash.
### Events
Events are a way for the __runtime__ to communicate information to the __debugger__.
All possible events are defined in `flow-storm.runtime.events`.
If there is nothing subscribed to runtime events the events will accumulate inside
`flow-storm.runtime.events/pending-events`, and as soon as there is a subscription they will be dispatched.
This is to capture events fired by recording when the __debugger__ is still not connected.
On the __debugger__ side they will accumulate on `flow-storm.debugger.events-queue` and will be dispatched by a
specialized thread. Most events are processed by `flow-storm.debugger.events-processor/process-event` but any part
of the __debugger__ can listen to __runtime__ events by adding a callback with `flow-storm.debugger.events-queue/add-dispatch-fn`.
### Tasks
Most of the __runtime__ functionality the __debugger__ calls is called synchronously, but there are some functions where doing
like this leads to suboptimal UX. This are most functionality that needs to search or collect on the timeline. For big recordings
this could take a long time.
For this reason, functions that loop on the timeline run under tasks. This are looping process that run async, can report progress
and can be interrupted.
All the functions that run tasks ends up in `-task`, like `search-collect-timelines-entries-task`.
From the __debugger__, for calling a task function `flow-storm.debugger.ui.tasks/submit-task` can be used.
On the __runtime__ side, there are a couple of utilities that make implementing this interruptible loopings easies :
- `flow-storm.runtime.debuggers-api/submit-batched-collect-interruptible-task` (for collecting functionality that traverse the entire timeline)
- `flow-storm.runtime.debuggers-api/submit-find-interruptible-task` (for looping through first match)
There
### Remote debugging
Remote debugging means that the __debugger__ and the __runtime__ run in different processes.
This is optional in Clojure but the only way of debugging in ClojureScript.
The difference with local debugging is that the interaction happens over a websocket and a repl connection instead of
direct function calls. Interaction here refers to api calls from the __debugger__ to the __runtime__ and events flowing
the other way around.
The debuggee can start a repl server which the __debugger__ can then connect to, while and the __debugger__ will start a
websocket server that the __runtime__ will connect to by running `flow-storm.runtime.debuggers-api/remote-connect`.
The reason there are two different kinds of connections between the same processes are capabilities and performance.
The repl connection is the only way of accomplishing some functionality in ClojureScript, while the websocket is
better suited for events dispatches.
Most of the api calls travel trough the websocket connection, you can see which goes through which connection in
`flow-storm.debugger.runtime-api/rt-api`.
## FlowStorm dev tips
The easiest way in my opinion to work with the FlowStorm codebase is by starting a repl with the `:dev` and `:storm`
aliases (unless one needs to specifically try Vanilla FlowStorm).
The FlowStorm codebase includes a dev namespace inside `src-dev/dev.clj`, which contains some handy code for development,
and is designed so you can work without having to restart the repl.
Specially :
* `start-local` to start everything locally, with spec checking the `flow-storm.debugger.state/state`
* `start-remote` to start only the __debugger__ and connect to a different process, with spec checking the `flow-storm.debugger.state/state`
* `stop` to shutdown the system gracefully
* `refresh` which will use tools.namespace to refresh namespaces
After the UI is running, everything you eval under the `dev-tester` namespace will be recorded. The `dev-tester`
namespace contains a bunch of meaningless functions, with the only purpose of generating data for testing the system.
In most situations, you should be able to change and re-eval a function on the repl and retry it without restarting the UI.
## Working with the UI
When tweaking the UI [ScenicView](https://github.com/JonathanGiles/scenic-view) helps a lot, since it allows us to
explore the JavaFX Nodes tree, and also reload css on the fly.
## Using FlowStorm as a backend for other tooling
FlowStorm includes an nRepl middleware `flow-storm.nrepl.middleware` which exposes all its functionality as nRepl
operations.
For examples of tooling using this middleware, take a look at [cider-storm](https://github.com/flow-storm/cider-storm).
## Working with ClojureStorm
Since ClojureStorm is a fork of Clojure, any technique you use for working with the Clojure compiler will apply here.
For example if you start the repl with `:storm`, `:dev` and `:jvm-debugger` you should be able to connect a Java debugger
like IntelliJ's one and add breakpoints. Then run whatever expression at the repl and step with the debugger.
Another handy tool for troubleshooting instrumentation is
[clj-java-decompiler](https://github.com/clojure-goes-fast/clj-java-decompiler).
Let's say you have instrumented:
```clojure
(defn sum [a b] (+ a b))
```
You can then decompile it into :
```java
public final class dev_tester$sum extends AFunction
{
public static Object invokeStatic(final Object a, final Object b) {
Number add;
try {
Tracer.traceFnCall(new Object[] { a, b }, "user", "sum", -1340777963);
Tracer.traceBind(a, "", "a");
Tracer.traceBind(b, "", "b");
Tracer.traceExpr(a, "3,1", -1340777963);
Tracer.traceExpr(b, "3,2", -1340777963);
add = Numbers.add(a, b);
Tracer.traceExpr(add, "3", -1340777963);
Tracer.traceFnReturn(add, "", -1340777963);
}
catch (Throwable throwable) {
Tracer.traceFnUnwind(throwable, "", -1340777963);
throw throwable;
}
return add;
}
@Override
public Object invoke(final Object a, final Object b) {
return invokeStatic(a, b);
}
}
```
or its bytecode equivalent.
## Working with ClojureScriptStorm
For working with ClojureScript storm we can use FlowStorm! since it is a Clojure codebase.
Take a look at this for ideas : https://jpmonettas.github.io/my-blog/public/compilers-with-flow-storm.html
## Installing local versions
You can install both artifacts locally by running :
```bash
$ VERSION=3.8.4-SNAPSHOT make install-inst # for building and installing the inst artifact
$ VERSION=3.8.4-SNAPSHOT make install-dbg # for building and installing the dbg artifact
```
================================================
FILE: docs/high_level_diagram.drawio
================================================
================================================
FILE: docs/index.html
================================================
================================================
FILE: docs/related-research-and-tools.md
================================================
- [Debugging Backwards in Time, Bil Lewis Paper](https://arxiv.org/pdf/cs/0310016.pdf)
- [Debugging Backwards in Time, Bil Lewis Debugger](https://github.com/OmniscientDebugger/LewisOmniscientDebugger)
- [Scalable Omniscient Debugging, Pothier, TOD, paper](https://pleiad.cl/papers/2007/pothierAl-oopsla2007.pdf)
- [Scalable Omniscient Debugging, Pothier, TOD, software](https://pleiad.cl/tod/)
- [NOD4J: Near-omniscient debugging tool for Java using size-limited execution trace](https://sel.ist.osaka-u.ac.jp/lab-db/betuzuri/archive/1199/1199.pdf)
- [Repeatedly-Executed-Method Viewer for Efficient Visualization of Execution Paths and States in Java](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=233a50cdc9ba4ce0b322053cebbc380ec93a728c)
- [JIVE: Java Interactive Visualization Environment](https://cse.buffalo.edu/jive/tutorials.html)
- [Undo debugger for Java](https://undo.io/solutions/products/java/)
- [ChrononSystems debugger for Java](https://chrononsystems.com/products/chronon-time-travelling-debugger)
- [Wallabyjs](https://wallabyjs.com/docs/intro/time-travel-debugger.html)
- [ZStep common lisp, video](https://www.youtube.com/watch?v=VzGj59hg2io)
- [ZStep common lisp, paper](https://www.lirmm.fr/%7Educour/Doc-objets/ECOOP/papers/0276/02760011.pdf)
================================================
FILE: docs/run_configs.drawio
================================================
================================================
FILE: docs/test-cases.org
================================================
* Test cases
| Name | Local Clj | Remote Clj | ClojureScript |
|---------------------------------+-----------+------------+----------------------|
| Basic #rtrace | OK | OK | OK |
| Basic #trace | OK | OK | OK |
| Def val | OK | OK | OK |
| Inspect val | OK | OK | OK |
| REPL [un]instrument var | OK | OK | OK / ~UN-FAIL-known~ |
| REPL [un]instrument namespaces | OK | OK | OK |
| Correct stepping | OK | OK | OK |
| Goto trace by typing idx | OK | OK | OK |
| Browser navigation | OK | OK | OK |
| Browser [un]instrument var | OK | OK | ~UN-FAIL-known~ |
| Browser [un]instrument ns | OK | OK | OK |
| Browser enable/disable instr | OK | OK | OK |
| Browser [un]instr sync | OK | OK | N/A |
| Cli-run | OK | N/A | N/A |
| Re-run flow | OK | OK | OK |
| Call tree | OK | OK | OK |
| Call tree goto trace | OK | OK | OK |
| Call search | OK | OK | OK |
| Functions list | OK | OK | OK |
| Functions list goto trace | OK | OK | OK |
| Fully instrument form | OK | OK | OK |
| UnInstrument forms from fn list | OK | OK | ~UN-FAIL-FAIL~ |
| Taps | OK | OK | OK |
| Themes | OK | OK | OK |
| Auto reconnect | N/A | OK | OK |
| | | | |
================================================
FILE: docs/timeline.drawio
================================================
================================================
FILE: docs/user_guide.adoc
================================================
= FlowStorm debugger User's Guide
:source-highlighter: rouge
:author: By Juan Monetta
:lang: en
:encoding: UTF-8
:doctype: book
:toc: left
:toclevels: 3
:sectlinks:
:sectanchors:
:leveloffset: 1
:sectnums:
_FlowStorm_ is a tracing debugger for Clojure and ClojureScript.
image::user_guide_images/intro_screenshot.png[]
It can instrument any Clojure code and provides many tools to explore and analyze your programs executions.
= Quick start
Before you start check _FlowStorm_ minimum requirements.
[IMPORTANT]
.Minimum requirements
====
- jdk >= 17 (if you still need to run it with jdk11 take a look at <<#_run_with_jdk_11,here>>)
- Clojure >= 1.10.0
====
== Clojure
There are two ways of using _FlowStorm_ for Clojure :
- With <<#_clojurestorm,ClojureStorm>> (recommended) : Swap your Clojure compiler at dev time by ClojureStorm and get everything instrumented automatically
- <<#_vanilla_flowstorm,Vanilla FlowStorm>> : Just add FlowStorm to your dev classpath and instrument by tagging and re-evaluating forms
=== ClojureStorm
This is the newest and simplest way of using _FlowStorm_. It requires you to swap your official Clojure compiler by _ClojureStorm_ only at dev time.
Swapping compilers sounds like a lot, but don't worry, _ClojureStorm_ is just a patch applied over the official compiler with some
extra stuff for automatic instrumentation. You shouldn't encounter any differences, it is only for dev, and you can swap it back
and forth by starting your repl with a different alias or lein profile.
The easiest way to run and learn _FlowStorm_ with _ClojureStorm_ is by running the repl tutorial.
==== Try it with no project and no config
You can start a repl with FlowStorm with a single command like this :
[%nowrap,bash]
----
;; on Linux and OSX
clj -Sforce -Sdeps '{:deps {} :aliases {:dev {:classpath-overrides {org.clojure/clojure nil} :extra-deps {com.github.flow-storm/clojure {:mvn/version "1.12.4"} com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}}}' -A:dev
;; on Windows
clj -Sforce -Sdeps '{:deps {} :aliases {:dev {:classpath-overrides {org.clojure/clojure nil} :extra-deps {com.github.flow-storm/clojure {:mvn/version """1.12.4"""} com.github.flow-storm/flow-storm-dbg {:mvn/version """4.5.9"""}}}}}' -A:dev
----
Pasting that command on your terminal will bring up a repl with _FlowStorm_ and the compiler swapped by _ClojureStorm_. When the repl comes up
evaluate the `:dbg` keyword to bring up the UI and then click on `Help->Tutorial` on the menu for a tour of the basics.
After the tutorial you may want to use it on your projects. You use it by adding a deps.edn alias or lein profile.
The simplest way is to setup it globally, so that is what we are going to do next. You can also add it only
to specific projects if they require special configurations.
==== Global setup as deps.edn aliases
You can setup your global `~/.clojure/deps.edn` (on linux and macOS) or `%USERPROFILE%\.clojure\deps.edn` (on windows) like this :
[%nowrap,clojure]
----
{...
:aliases
{:1.12-storm {:classpath-overrides {org.clojure/clojure nil}
:extra-deps {com.github.flow-storm/clojure {:mvn/version "1.12.4"}
com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}
;; Optional plugins you find yourself using regularly
:fs-web-plugin {:extra-deps {com.github.flow-storm/flow-storm-web-plugin {:mvn/version "1.0.0-beta"}}
:jvm-opts ["-Dclojure.storm.instrumentOnlyPrefixes.webPlugin=org.httpkit.server,ring.adapter.jetty,next.jdbc.result-set"
"-Dflowstorm.plugins.namespaces.webPlugin=flow-storm.plugins.web.all"]}
...}}
----
Then you can start your repls with the `:1.12-storm` alias (like `clj -A:1.12-storm`). When the repl comes up evaluate the `:dbg` keyword to bring up the UI,
then click on `Help->Tutorial` on the menu for a tour of the basics.
==== Global setup as leiningen profiles
You can setup your global `~/.lein/profiles.clj` (on linux and macOS) or `%USERPROFILE%\.lein\profiles.clj` (on windows) like this :
[%nowrap,clojure]
----
{:1.12-storm
{:dependencies [[com.github.flow-storm/clojure "1.12.4"]
[com.github.flow-storm/flow-storm-dbg "4.5.9"]]
:exclusions [org.clojure/clojure]}
;; Optional plugins you find yourself using regularly
:fs-web-plugin
{:dependencies [[com.github.flow-storm/flow-storm-web-plugin "1.0.0-beta"]]
:jvm-opts ["-Dclojure.storm.instrumentOnlyPrefixes.webPlugin=org.httpkit.server,ring.adapter.jetty,next.jdbc.result-set"
"-Dflowstorm.plugins.namespaces.webPlugin=flow-storm.plugins.web.all"]}
...}
----
Then you can start your project repls with `+1.12-storm` profile (like `lein with-profile +1.12-storm repl`). When the repl comes up evaluate the `:dbg` keyword to bring up the UI,
then click on `Help->Tutorial` on the menu for a tour of the basics.
[NOTE]
.Running lein repl without a project
====
For some reason if you run `lein with-profile +1.12-storm repl` outside of a project it will not run with the profile
activated correctly.
====
==== Per project setup with deps.edn
If your project is using deps.edn, you can add an alias that looks like this :
[%nowrap,clojure]
----
{...
:aliases {:1.12-storm
{;; for disabling the official compiler
:classpath-overrides {org.clojure/clojure nil}
:extra-deps {com.github.flow-storm/clojure {:mvn/version "1.12.4"}
com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}}}
----
Once you have setup your deps.edn, start your repl with the `:1.12-storm` alias and run the debugger by evaluating
the `:dbg` keyworkd on your repl (this means just type `:dbg` and hit return).
If it is your first time using FlowStorm, when the UI comes up click on `Help->Tutorial` on the menu for a tour of the basics.
If you need more fine control over instrumentation see <<#_controlling_instrumentation,controlling instrumentation>>.
==== Setup with leiningen
If your project uses lein, you can add a profile that looks like this :
[%nowrap,clojure]
----
(defproject my.project "1.0.0"
:profiles {:1.12-storm
{:dependencies [[com.github.flow-storm/clojure "1.12.4"]
[com.github.flow-storm/flow-storm-dbg "4.5.9"]]
:exclusions [org.clojure/clojure]}}
...)
----
Once you have setup your lein profile globally or per project, start your repl with the `1.12-storm` profile and run the debugger by evaluating
the `:dbg` keyworkd on your repl (this means just type `:dbg` and hit return).
Make sure you activate the profile with `lein with-profile +1.12-storm repl`.
If it is your first time using FlowStorm, when the UI comes up click on `Help->Tutorial` on the menu for a tour of the basics.
If you need more fine control over instrumentation see <<#_controlling_instrumentation,controlling instrumentation>>.
[NOTE]
.lein dependencies
====
If you are using lein < 2.11.0 make sure your global :dependencies don't include the official org.clojure/clojure dependency.
Moving to lein latest version should work ok even if your global :dependencies contains the Clojure dep.
====
=== Vanilla FlowStorm
If you use the https://clojure.org/guides/deps_and_cli[clojure cli] you can start a repl with the _FlowStorm_ dependency loaded like this :
[,bash]
----
;; on Linux and OSX
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}'
;; on Windows
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version """4.5.9"""}}}'
----
If you are a https://leiningen.org/[lein] user add the dependency to your project.clj `:dependencies` and run `lein repl`.
Then require the api namespace and start the debugger :
[%nowrap,clojure]
----
user> (require '[flow-storm.api :as fs-api]) ;; the only namespace you need to require
user> (fs-api/local-connect) ;; will run the debugger GUI and get everything ready
----
You should now see a empty debugger window. Click on the recording button to leave the debugger in
recording mode and the let's debug something:
[%nowrap,clojure]
----
user> #rtrace (reduce + (map inc (range 10))) ;; #rtrace will instrument and run some code
----
After running it, you should get the return value of the expression (as if #rtrace wasn't there),
but now you will also have the debugger UI showing your recordings.
From here you probably want to check out the <<#_flows_tool, Flows tool>> which contains a lot of information
about exploring your recordings.
== ClojureScript
Debugging ClojureScript is a case of remote debugging in _FlowStorm_. This means the debugger
will run in a separate process and connect to the debuggee (your browser or nodejs runtime) via a websocket and optionally
an nrepl server.
There are two ways of using _FlowStorm_ with ClojureScript :
- With <<#_clojurescriptstorm_with_shadow_cljs,ClojureScriptStorm>> (recommended) : Swap your ClojureScript compiler by ClojureScriptStorm at dev and get everything instrumented automatically
- <<#_clojurescript_vanilla_flowstorm,Vanilla FlowStorm>> : Just add FlowStorm to your dev classpath and instrument by tagging and re-evaluating forms
_ClojureScriptStorm_ is a fork of the official ClojureScript compiler that adds automatic instrumentation so you don't need to think about it (you can still disable it when you don't need it).
You use it by swapping the official ClojureScript compiler by _ClojureScriptStorm_ at dev time, using dev aliases or profiles.
[NOTE]
.Repl connection
====
For enabling every debugger feature, _FlowStorm_ needs to connect to a cljs repl.
Currently only shadow-cljs repl over nrepl is supported.
====
=== ClojureScriptStorm with shadow-cljs
[IMPORTANT]
.Minimum requirements
====
- For ClojureScript 1.11.* shadow-cljs >= 2.25.4, For ClojureScript 1.12.* shadow-cljs >= 3.1.1
- FlowStorm >= 3.7.4
====
For setting up _FlowStorm_ with shadow-cljs you need to modify two files, your `shadow-cljs.edn` and your `deps.edn`.
This is setup once and forget, so once you have configured _FlowStorm_ you can do everything from the UI, without
any other sources modifications.
If you want a shadow-cljs template to play with, take a look at https://github.com/jpmonettas/shadow-flow-storm-basic/[this repo].
[NOTE]
.shadow-cljs
====
Currently you can only use _ClojureScriptStorm_ with shadow-cljs if you are resolving your
dependencies with deps.edn. This means having `:deps true` or similar in your shadow-cljs.edn.
If you have your dependencies directly in your shadow-cljs.edn you will have to use <<#_clojurescript_vanilla_flowstorm,Vanilla FlowStorm>>
for now.
This is because there is currently no way to swap the ClojureScript compiler in shadow-cljs.edn.
====
First, make your shadow-cljs.edn looks something like this :
[%nowrap,clojure]
----
{:deps {:aliases [:1.12-cljs-storm]}
:nrepl {:port 9000}
...
:builds
{:my-app {...
:devtools {:preloads [flow-storm.storm-preload]
:http-port 8021}}}}
----
So, the important parts are: you need to tell shadow to apply your deps.edn :1.12-cljs-storm alias, set up a nrepl port,
and also add `flow-storm.storm-preload` to your preloads. If you have other preloads make sure `flow-storm.storm-preload`
is the first one.
Then, modify your `deps.edn` dev profile to look like this :
[%nowrap,clojure]
----
{...
:aliases
;; this alias can be defined globally in your ~/.clojure/deps.edn so you don't have to modify this file in your project
{:1.12-cljs-storm
{:classpath-overrides {org.clojure/clojurescript nil} ;; disable the official compiler
:extra-deps {thheller/shadow-cljs {:mvn/version "3.3.4"
:exclusions [org.clojure/clojurescript]}
;; bring ClojureScriptStorm
com.github.flow-storm/clojurescript {:mvn/version "1.12.134-3"}
;; add FlowStorm runtime dep
com.github.flow-storm/flow-storm-inst {:mvn/version "4.5.9"}}}}}
----
There are lots of things going on there, but the main ones are: disabling the official compiler, adding
_ClojureScriptStorm_ and _FlowStorm_ dependencies, and then configuring what you want _ClojureScriptStorm_ to automatically
instrument.
By default the JVM property `cljs.storm.instrumentAutoPrefixes` is true so all your project top level namespaces
will be instrumented automatically.
If you need to set that property to false it is important to configure what namespaces you want to instrument,
and you do this by setting the `cljs.storm.instrumentOnlyPrefixes` jvm property.
This is a comma separated list of namespaces prefixes, you normally want your app namespaces plus some libraries, like :
`cljs.storm.instrumentOnlyPrefixes=org.my-app,org.my-lib,hiccup`
And this is it. Once you have it configured, run your shadow watch as you normally do, load your app on the browser (or nodejs).
Whenever your need the debugger, on a terminal run the ui with your shadow-cljs.edn data :
[,bash]
----
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-app
----
and then reload you page so it connects to it.
Since we started the app with `flowstorm.startRecording=false` you will have to click on the record button once to start recording.
Whenever recording is enable and something executes under an instrumented namespace you should see the recordings appear in the debugger
under the main thread.
[NOTE]
.recording expressions typed on the repl
====
If you type at the repl something like `(defn foo [a b] (+ a b))` under an instrumented ns, the `foo` function will get instrumented
automatically and you will able to explore the recordings after the function is called.
On the other side, typing a simple expression like `(+ 1 2)` will not show anything, this is currently a limitation but you can
still make that work by wrapping the expression on a fn and immediately calling it, like `((fn [] (+ 1 2)))`
====
=== ClojureScriptStorm with cljs.main
You can use _FlowStorm_ and _ClojureScriptStorm_ with cljs.main.
The easiest way to try it is just by starting a repl, like this :
[%nowrap,bash]
----
clj -Sforce -J-Dcljs.storm.instrumentOnlyPrefixes=cljs.user -Sdeps '{:deps {com.github.flow-storm/clojurescript {:mvn/version "1.12.134-3"} com.github.flow-storm/flow-storm-inst {:mvn/version "4.5.9"}}}' -M -m cljs.main -co '{:preloads [flow-storm.storm-preload]}' --repl
----
If you run the command above you are running cljs.main --repl which will start a ClojureScript repl on
your terminal and open a browser connected to it. You runtime will also start with FlowStorm preloaded and everything under `cljs.user`
is going to be instrumented.
Then on a different terminal run the _FlowStorm_ UI :
[%nowrap,bash]
----
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger
----
And now refresh your browser page so your browser app connects to the UI.
[NOTE]
.Limitations
====
There are some small limitations like not being able to modify instrumentation from the UI without restarting the repl.
This is because the _FlowStorm_ UI needs to also connect via nrepl to JVM process running the compiler, which isn't available
when running cljs.main.
====
=== ClojureScript vanilla FlowStorm
Let's say you are using https://github.com/thheller/shadow-cljs[shadow-cljs] to start a ClojureScript repl.
First you need to add _FlowStorm_ dependency to your project dependencies, like this :
[%nowrap,clojure]
----
$ cat shadow-cljs.edn
{...
:dependencies [... [com.github.flow-storm/flow-storm-inst "4.5.9"]]
;; the next two lines aren't needed but pretty convenient
:nrepl {:port 9000}
:my-build-id {:devtools {:preloads [flow-storm.preload]}}
...}
----
Then let's say you start your repl like :
[,bash]
----
npx shadow-cljs watch :my-build-id
shadow-cljs - config: /home/jmonetta/demo/shadow-cljs.edn
shadow-cljs - server version: 2.19.0 running at http://localhost:9630
shadow-cljs - nREPL server started on port 9000
shadow-cljs - watching build :my-build-id
[:my-build-id] Configuring build.
[:my-build-id] Compiling ...
[:my-build-id] Build completed. (127 files, 0 compiled, 0 warnings, 6.19s)
cljs.user=>
----
As you can see from the output log shadow-cljs started a nrepl server on port 9000, this is the port _FlowStorm_ needs to connect to,
so to start the debugger and connect to it you run :
[,bash]
----
;; on linux and mac-os
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-build-id
;; on windows
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version """4.5.9"""}}}' -X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-build-id
----
And that is all you need, the debugger GUI will pop up and everything will be ready.
Try tracing some code from the repl :
[%nowrap,clojure]
----
cljs.user> #rtrace (reduce + (map inc (range 10))) ;; #rtrace will instrument and run some code
----
After running it, you should get the return value of the expression (as if #rtrace wasn't there).
The debugger thread list (the one on the left) shows all the threads it has recordings for. Because we are
in javascript land there will always be just one thread, called `main`.
Double clicking it should open the "thread exploring tools" for that thread in a new tab.
This guide will cover all the tools in more detail but if you are interested in code stepping for example you will find
it in the `code stepping tool` at the bottom left corner of the thread tab, the one that has the `()` icon.
Click on it and use the stepping controls to step over the code.
Now that everything seems to be working move on and explore the many features _FlowStorm_ provides. There are many ways of instrumenting
your code, and many ways to explore its executions.
If you are not using a repl or the repl you are using isn't supported by _FlowStorm_ yet you can still use the debugger
but not all features will be supported (mainly the browser features).
For this you can start the debugger like before but without any parameters, like this :
[,bash]
----
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger
----
And then go to your app code and call `(flow-storm.runtime.debuggers-api/remote-connect)` maybe on your main, so every time your program starts
will automatically connect to the repl.
[NOTE]
.ClojureScript environments
====
_FlowStorm_ is supported for ClojureScript in :
- Browsers
- NodeJS
- React native
====
[NOTE]
.NodeJs and react-native
====
On NodeJs and react-native you need to install the `websocket` library.
Do this by running `npm install websocket --save`
For react-native if your app is running inside a cellphone you will have to also provide the `:debugger-host` key
to `flow-storm.debugger.main/start-debugger` with your box ip address, unless you are using adb reverse with your ports for
which you will have to `adb reverse tcp:7722 tcp:7722` (the debugger websocket port)
====
[NOTE]
.App initialization debugging
====
If you need to debug some app initialization, for adding `#trace` tags before the debugger is connected you
will have to require flow-storm.api yourself, probably in your main. All the tracing will be replayed to the debugger
once it is connected.
====
Here is a repo you can use if you want to try _FlowStorm_ with shadow-cljs https://github.com/flow-storm/shadow-flow-storm-basic
=== Multiple ClojureScript builds
You can setup FlowStorm to debug multiple ClojureScript builds. This can be useful when your application is made up of multiple parts,
like when you have web workers.
Debugging multiple builds require multiple debugger instances, one per build.
The FlowStorm UI will start a websocket server, by default on 7722, so if you want to run multiple instances of it, you need
to run each instance under a different port. You can do this by providing a `:ws-port` to the startup command.
So let's say you want to run two debuggers, one for your page and one for a webworker, your can run them like this :
[,bash]
----
# on one terminal start your app debugger instance
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-app :ws-port 7722
# on a second terminal start your webworker debugger instance
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-web-worker :ws-port 7733
----
Now you also need to configure your builds to tell them what port they should connect to.
You do this by writing different preloads for each of your builds, and then using them instead of your `flow-storm.storm-preload`, like:
`my_app.main_storm_preload.cljs`
[%nowrap,clojure]
----
(ns my-app.main-storm-preload
(:require [cljs.storm.tracer]
[flow-storm.tracer :as tracer]
[flow-storm.runtime.debuggers-api :as dbg-api]))
(dbg-api/start-runtime)
(tracer/hook-clojurescript-storm)
(dbg-api/remote-connect {:debugger-host "localhost" :debugger-ws-port 7722})
----
`my_app.webworker_storm_preload.cljs`
[%nowrap,clojure]
----
(ns my-app.webworker-storm-preload
(:require [cljs.storm.tracer]
[flow-storm.tracer :as tracer]
[flow-storm.runtime.debuggers-api :as dbg-api]))
(dbg-api/start-runtime)
(tracer/hook-clojurescript-storm)
(dbg-api/remote-connect {:debugger-host "localhost" :debugger-ws-port 7733})
----
They are the same as `flow-storm.storm-preload` just with different port numbers.
Now you can configure your shadow-cljs.edn like this :
[%nowrap,clojure]
----
{...
:builds
{:app
{:target :browser
...
:modules
{:my-app {:init-fn my.app/init
:preloads [my-app.main-storm-preload]}
:my-webworker {:init-fn my.app.worker/init
:preloads [my-app.webworker-storm-preload]
:web-worker true}}}}}
----
[NOTE]
.Multiple debuggers tips
====
You can change the theme or customize the styles of different instances to make it easier to
know which debugger instance is connected to which application.
====
== Babashka
You can debug your babashka scripts with FlowStorm using the JVM. The process is quite simple.
Let's say we want to debug this example script https://raw.githubusercontent.com/babashka/babashka/master/examples/htmx_todoapp.clj
which runs a webserver with a basic todo app.
First we need to generate a deps.edn by running `bb print-deps > deps.edn`
Then modify the resulting deps.edn to add the FlowStorm alias like this :
[%nowrap,clojure]
----
{...
:aliases {:dev {:classpath-overrides {org.clojure/clojure nil} ;; for disabling the official compiler
:extra-deps {com.github.flow-storm/clojure {:mvn/version "1.12.4"}
com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}
:jvm-opts ["-Dclojure.storm.instrumentOnlyPrefixes=user"]}}}
----
With `clojure.storm.instrumentOnlyPrefixes=user` we are telling ClojureStorm to instrument everything inside
the `user` namespace since the script doesn't contain any namespace declaration.
And that is it, you can now start your clojure repl as usual, with `clj -A:dev` and then eval the `:dbg` keyword to
start the debugger UI.
Then eval the entire file to compile everything. To start the server in this example you will have to remove the wrapping
that is basically only allowing the server to run if we are running from babashka, like this :
[%nowrap,clojure]
----
(when true #_(= *file* (System/getProperty "babashka.file"))
...)
----
so we can also start it from Clojure.
After the server has started, you can use the app from the browser and everything will get recorded as usual.
= The tool bar
The toolbar as well as the menu provides quick access to some general commands.
image::user_guide_images/toolbar.png[]
From left to right:
- Cancel current running task. Whenever you a running a task that can take some time, this button will be red, and you can use it to cancel the task.
- The `Inst enable` button allows to enable/disable instrumentation when in a Storm environment. A change on instrumentation will only affect newly compiled code.
= Flows tool
The `Flows` vertical tab contains a bunch of tools for recording and analyzing your programs executions.
First of all, what are Flows?
A Flow is an "execution flow" recording unit. The only purpose of a flow is to group recording activity.
This grouping allows us for example to run some code and record it under `flow-0`, then modify our code, run it again, and
record this second run (or flow) under `flow-1`. Now we can access both recordings separately.
image::user_guide_images/recording_controls.png[]
When you first open FlowStorm UI you will see four things, from left to right :
- Clear your recordings if any.
- Start/Stop recording. You can keep your heap from growing by stopping recording when you don't need it.
- Start/Stop recording the multi-thread timeline. Check out the <<#_multi_thread_timeline, multi-thread timeline>> tool.
- The `Rec on` combo-box to select under what flow new recordings are going to be stored.
Whenever there is something recorded for a flow, a new tab with the flow name will appear.
Execution inside a flow will be grouped by threads. So the first thing you will see on a flow is a menu of threads
we have recordings for so far. This threads will be referred sometimes as timelines, since they are a sequence of
recorded execution steps.
Let's say for example we have selected to record under `flow-1` and run some multi threaded code.
We are going to see something like this :
image::user_guide_images/multi_flows_1.png[]
There is a lot going on in the screenshot above, but the most important are :
- we have configured FlowStorm to record new executions under `flow-1`
- we have recorded stuff under `flow-1` and there are also some previous recordings under `flow-0`
- we are currently looking at `flow-1`, we have opened to explore the thread with id `1` called `main` and we are exploring it in <<#_code_stepping,the code stepper>>
- `Threads [4]` indicates we have recorded activity in 4 threads, which we can access via this menu
Now for a different example :
image::user_guide_images/multi_flows_2.png[]
This second image shows us exploring the recordings of a thread with id `474`, called `pool-4-thread-4` on `flow-0`.
image::user_guide_images/flows_toolbar.png[]
The `Flows tool` also contains a toolbar that contains the Quick jump box.
Use it for quickly opening the first recording of a function in <<#_code_stepping,the code stepper>>.
Will autocomplete the first 25 matches.
In the screenshot above we see analyzing the recordings in <<#_code_stepping,the code stepper>> but there are many tools to explore the recorded timelines,
which we are going to describe next.
== Code tool
image::user_guide_images/code_tool_tab.png[]
The code tool is the first of the `Flows` tab. It provides most of the functionality found in a traditional debugger.
You can use it to step over each expression, visualize values, locals and more.
=== Code stepping
The code tool allows you to step and "travel throught time" in two ways:
- Use the controls at the top to step over your code in different ways.
- Click on the highlighted forms to position the debugger at that point in time.
image::user_guide_images/controls.png[]
For moving around using the controls we have two rows of buttons.
The second row of controls, the most important one, are the stepping controls.
From left to right they are :
- Step over backwards, will make one step backwards always staying on the same frame.
- Step backwards, will step backwards in time going into sub functions.
- Step out, will position the debugger in the next step after this function was called.
- Step forward, will step forward in time going into sub functions.
- Step over forward, will make one step forwards always staying on the same frame.
The numbers at the center show `current_step_index / total_steps`. This means that a total of `total_steps` has been recorded
for this thread so far. Write any number (less than total_steps) on the text box to jump into that position in time.
The buttons around the step counter are :
- Jump to the first step of the recording.
- Jump to the last step of the recording.
On the first row we have more controls, also for moving around in time.
From left to right we have :
- Undo navigation
- Redo navigation
- Add a <<#_bookmarks, bookmark>>
- The last stepping controls to the right are the <<#_power_stepping, power stepping>> controls.
[NOTE]
.Highlighting
====
Only the forms that were executed at least once for the current function frame will be highlighted.
====
This means that code can be un-highlighted for two reasons:
- there isn't any recording for that part of the code
- there is a recording but doesn't belong to this function frame.
image::user_guide_images/stepper_highlighting.png[]
In the contrived example above we see we are stepping the `foo` function. All inside this function
body is highlighted but the bodies of the two anonymous functions for mapping and reducing. This
will only get highlighted once you step into their bodies.
In this case you are sure there are recordings for these functions bodies because the reduce is
non lazy, so if you keep stepping eventually you will get into their bodies, but there is a faster way.
image::user_guide_images/stepper_highlighting_2.png[]
For this you can right click on any un-highlighted expression that you think there could be a recording for and
select `Jump forward here`.
image::user_guide_images/stepper_highlighting_3.png[]
This will make FlowStorm scan from the current point of the timeline searching forward for a value
recorded at that coordinate (if any) and move the stepper to that point in time.
You also have `Jump to first record here` which will scan from the beginning of the timeline and `Jump backwards here`
which will search backwards from the current position.
=== Power stepping
image::user_guide_images/controls_power_custom.png[]
The controls at the right are power stepping controls. They provide more powerfull ways of stepping through the code.
Clicking on the first, back, next or last buttons will navigate the timeline using the selected power stepping tool in the dropdown.
There are currently six power stepping tools :
- `identity`, will step to the prev/next value which identity is the same as the current value.
- 'equality', will step to the prev/next value which is equals (clojure equality) to the current value.
- `same-coord` will step to the prev/next value for the same coordinate. This means it will move to the next recording in
the timeline for this exact place in the code you are currently in. You can also see it as take me to all the situations
when the current expression executed doesn't matter how we got to it.
- `custom`, allows you to provide a predicate, which will be used to find the next step.
If you define it like `(fn [v] (map? v))` will make the power stepper step over all map values.
- `custom-same-coord`, the same as `custom` but fixed on the current coordinate like `same-coord`.
- `identity-other-thread`, will step to a position which identity is the same as the current value in a different thread.
Here the prev and next arrows do the same thing, it will just jump to the first position that matches this value on a
different thread. This has some limitations. If there are more than two threads working with this identity there is no way
of choosing which thread to go. If you need more control, checkout the <<#_programmable_debugging,programmable debugging>>
section, specially the `find-expr-entry` function.
- `fn-call`, allows you to provide a function to step to.
[NOTE]
.Custom stepping
====
Custom power stepping is only supported in Clojure now.
====
Power stepping automatically skips all values equals to `:flow-storm.power-step/skip`. This can be useful when combined
with <<#_dealing_with_mutable_values, snapshot-value>> as a way of ignoring some of them, which provides a way of sampling
tight loops like in games.
=== Searching
image::user_guide_images/search_access.png[]
You can use the search tool to search over all your flow recorded expressions and then make the stepper jump to them.
You can find the search tool under `More tools -> Search`.
There are multiple ways of searching:
- By pr-str
- By data window current value
- By predicate
==== Searching by pr-str
image::user_guide_images/search_pr_str.png[]
This type of search will walk over the selected threads expressions, converting their values to strings with `pr-str` up to the selected level and depth
and then checking if the resulting string contains your provided query string.
==== Searching by DataWindow value
image::user_guide_images/search_data_window.png[]
Searching by data window value allows you to select any of the current data windows and will search for the current selected data window value
over the selected threads expressions values using identity.
==== Searching by predicate
image::user_guide_images/search_pred.png[]
Searching by predicate allows you to provide a Clojure predicate which will be used over all selected threads expressions values.
=== Loops
Whenever you click a highlighted form that has been executed multiple times inside the same function call (any kind of loop),
instead of immediately jumping into it, FlowStorm will popup a menu, like in the picture below :
image::user_guide_images/loops.png[]
This is the loops navigation menu. It allows you to quickly move around interesting iterations of the loop.
The menu will display slightly different options depending on you current position. The `[FIRST] ...` and `[LAST] ...`
entries will always show, which allows you to quickly jump to the first and last iteration of the loop.
If you are currently before the loop, clicking into any expression inside the loop will show the first 20
values for the clicked expression.
If instead you are currently in a expression after the loop, clicking back to an expression inside the loop,
will show the last 20 values for the clicked expression.
Now if you are currently stepping inside the loop, clicking any other expression inside it will show you 10 values
before and 10 values after of your current position.
Clicking on any of this entries will take you to that position in time.
If this is not enough, and you want to see all the values taken by some expression along the loop, you can always
use the <<#_printer, printer tool>>.
=== Exceptions debugging
`FlowStorm` will capture all functions that didn't return because an exception unwind the stack, even
when that exception was captured further and it didn't bubble up.
image::user_guide_images/exceptions.png[]
When an unwind situation is recorded a combobox will show up in the toolbar, containing the functions names
together with the exceptions types. If you hover the mouse over any of them, a tooltip will display the exception message.
Clicking on any of them will position the stepper at that point in time so you can explore what happened before.
You can configure FlowStorm to automatically jump to exceptions with the `Config` menu by checking `Auto jump to exception`
which is disabled by default.
=== Locals
The locals panel will show the locals visible for the current point in time and their values at binding time.
image::user_guide_images/locals.png[]
Right clicking on them will show a menu where you can :
- define all
- define the value with a name, so you can use it at the repl
- inspect the value with a <<#_data_windows,data window>>
- tap the value as with `tap>`
`Define all` will define all the bindings currently visible in the locals pane in the current form namespace.
This is useful for trying things at your editor as described here https://www.cognitect.com/blog/2017/6/5/repl-debugging-no-stacktrace-required
[NOTE]
.Locals and mutable values
====
The Locals pane will show the value of each binding for a symbol at binding time, which are the same thing
no matter where you are in the current block when working with immutable objects, but not when working with mutable ones.
If what was bound was muttable in any way, you will be seeing the value at binding time, and not at current time.
====
=== Stack
The stack panel will always show the current stacktrace. Be aware that the stacktrace
only include functions calls that had been recorded, so if you aren't recording everything
there will be gaps.
image::user_guide_images/stack.png[]
Double clicking on any of the stack entries will make the debugger jump to that point in time.
=== Value panels
Value panels show in many places in _FlowStorm_.
image::user_guide_images/value_panels2.png[]
The value panel in the code tool always display a pretty print of the current expression value.
You can configure the print-level and print-meta for the pretty printing by using the controls at the top.
The value panel showing the current expression in the code stepper is a little bit special since it also
contains a <<#_data_windows,data window>> tab which allows you to quickly navigate the value or give it custom
visualizations.
image::user_guide_images/value_panels1.png[]
==== Define value for repl
Use the `def` button to define a var pointing to the current inspector value.
You can use / to provide a namespace, otherwise will be defined under [cljs.]user
=== Goto to file:line
Clicking on the `Actions->Goto file:line` menu allows you to search and jump to the first recording of a expression
with a file and line, given that one exists.
It will ask you for a file and line in the format of `:`.
If you have a file like `src/org/my_app/core.clj` and you are interested in expressions evaluating on like 42
you should search like `org/my_app/core.clj:42`.
== Call Stack tree tool
The call stack tree tool is the second one of the `Flows` tab. It allows you to see the execution flow by expanding its call stack tree.
image::user_guide_images/callstack_tool_tab.png[]
The call stack tree is useful for a high level overview of a complex execution and also as a tool for quickly moving through time.
You can jump to any point in time by double clicking on a node or by right clicking and on the context menu selecting `Step code`.
image::user_guide_images/callstack_tree.png[]
Use the button at the top left corner of the tree tool to show the current frame of the debugger in the tree.
There are also two <<#_value_panels,value panels>> at the bottom that show the arguments and return value for the currently selected function call.
[NOTE]
.Disabling the call stack tree tool
====
The call stack tree tool can be enable/disable on the fly if you are not using it and performance is an issue,
since keeping it updated can be expensive.
You can disable it from the Config menu or via the `flowstorm.callTreeUpdate=false` JVM prop.
====
== Functions tool
The functions tool is the third one of the `Flows` tab.
image::user_guide_images/functions_tool_tab.png[]
It shows a list of all traced functions sort by how many times the have been called.
image::user_guide_images/functions_calls.png[]
Normal functions will be colored black, multimethods magenta and types/records protocols/interfaces implementations in green.
Together with the <<#_call_stack_tree_tool, call stack tree>> it provides a high level overview of a thread execution, and allows you to
jump through time much faster than single stepping.
You can search over the functions list by using the bar at the top.
=== Function calls
Clicking on the calls counter of any function will display all function calls on the right sorted by time.
Each line will show the arguments vector for each call, and their return value.
Use the check boxes at the top to hide some of the arguments.
image::user_guide_images/function_calls.png[]
Double clicking on any row in the functions call list will jump to the stepper at that point in time.
You can also use the `args` and `ret` buttons to open the values on the inspector.
== Multi-thread timeline
You can use this tool to record, display and navigate a total order of your recordings in a timeline.
This can be used, for example, to visualize how multiple threads expressions interleave, which is sometimes useful to debug race conditions.
You enable/disable the multi-thread timeline recording using its button on the toolbar. Recording on the multi-thread
timeline will make your program execution a little slower so it is recommended to have it paused unless you need it.
When you have something recorded on the multi-thread timeline you access the tool from the top right corner.
image::user_guide_images/multi_timeline_access.png[]
As an example, let's say you record the execution this function :
[,clojure]
----
(defn run-parallel []
(->> (range 4)
(pmap (fn [i] (factorial i)))
(reduce +)))
----
By opening the tool a window like this should pop up :
image::user_guide_images/timeline.png[]
As you can see the timeline tool displays a linear representation of your expressions. Times flows from top to bottom and
each thread gets assigned a different color. Every time a function is called or returns you will see it under the `Function`
column, and for each expression executed you will see a row with its `Expression` and `Value`.
Double clicking any row will make your code stepper (on the main window) jump to the code at that point in time.
[NOTE]
.Big recordings timeline
====
Rendering the timeline needs some processing to render each sub-form and print each value so be aware it could be slow
if you try it on big recordings.
====
There is also a `Only functions?` checkbox at the top that will retrieve only function calls and can be used to visualize
the threads interleaving at a higher level.
== Printer
_FlowStorm_ has a lot of functionality to replace printing to the console as a debugging method since most of the time it is pretty
inefficient. Nonetheless, sometimes adding a bunch of print lines to specific places in your code is a very powerful way
of understanding execution.
For this cases _FlowStorm_ has the `Printer tool`, which allows you to define, manage and visualize print points, without the need
of re running your code. It will work on your recordings as everything else.
You can add and re run print points over your recordings as many times as you need. To add a print point, just right click on any
recorded expression.
image::user_guide_images/printer_add.png[]
It will ask you for a couple optional fields.
image::user_guide_images/printer_add_box.png[]
The `Message format` is the "println text". A message to identify the print on the printer output. Here you can use any text, in which you can
optionally use `%s` for the printed value, same as you would use it with format.
The `Expression` field can be use to apply a transformer function over the value before printing it. Useful when you want to see a part of the value.
image::user_guide_images/printer_access.png[]
After you add them, you can access the `Printers tool` by navigating to `More tools -> Printers`.
The threads selector allows you to select the thread the prints are going to run on.
Leaving it blank will run prints over all threads recordings (checkout the notes for caveats).
Clicking the `refresh` button will [re]run the printing again over the current recordings.
image::user_guide_images/printer.png[]
You can tweak your prints at any time, like changing the print-length, print-level, message, transform-fn or just temporarily disable any of them.
When you are ok re-setting you prints, just click refresh and they will print again.
Double clicking on any printed line will jump to the Flows code tab, with the debugger pointed to the expression that generated the print.
[IMPORTANT]
.Multi-thread prints order
====
If you select `All` threads, and have a multi-thread timeline recording, then the printer will use it and you can use prints to debug threads
interleaving for example, but if you run your printers with `All` threads selected without a multi-thread timeline recording they will print
sorted by thread and not in the order they happened.
====
== Bookmarks
Bookmarks are another quick way of jumping around in code and they can be added from your code or the FlowStorm UI.
You can find you bookmarks on the top menu `View -> Bookmarks`.
image::user_guide_images/bookmarks.png[]
Double clicking on any bookmark will make the debugger jump back to its position.
=== Code bookmarks
You add code bookmarks by adding the `(bookmark)` statement to your code, which optionally accepts a label.
The first time a bookmark statement is executed it will make the FlowStorm UI jump to it. Since this behavior
is similar to a `debugger` statement in languages like Javascript, it is also aliased as `(debugger)` so you can
use whichever you prefer.
[NOTE]
.ClojureScript support
====
This is currently only supported when using ClojureScriptStorm >= 1.11.132-9
====
=== UI bookmarks
UI bookmarks are useful when you find yourself jumping around, trying to understand a complex execution. They enable
you to mark execution positions so you can come back to them later.
image::user_guide_images/bookmarks_add_btn.png[]
You can bookmark the current position by pressing the bookmark button in the code tool, next to your stepping controls.
It will ask you the bookmark description.
= Browser tool
The browser tool is pretty straight forward. It allows you to navigate your namespaces and vars,
and provides ways of <<#_controlling_instrumentation,managing what gets instrumented>>.
image::user_guide_images/browser.png[]
= Outputs tool
image::user_guide_images/outputs.png[]
The outputs tool can be used instead of your normal IDE/Editor panel to visualize your evaluations
results, your taps outputs and your `*out*` and `*err*` streams writes (like printlns).
The advantages being :
- Custom visualizations
- Quick nested values navigation
- Quick taps values navigation
- Datafy nav navigation
- Access to all previously tapped values
- Access to the last 10 evaluated values (instead of just `*1` and `*2`)
- Ability to search tapped values in Flows
The taps visualization system works out of the box while the evals result and printing capture currently
depends on you using nrepl and starting with the flow-storm middleware. Checkout the outputs setup
section for instructions.
[NOTE]
.ClojureScript support
====
Only the taps viewer is currently supported on ClojureScript. The last evaluations
and the out and err streams capture aren't supported yet.
====
== Middleware setup
For using all the features in the Outputs tool you need to be using nrepl and start your repl with
`flow-storm.nrepl.middleware/wrap-flow-storm` middleware.
If you use Cider for example you can add it to `cider-jack-in-nrepl-middlewares` via customizing the global
value or by using `.dir-locals.el`.
== Output data window
The top panel is a <<#_data_windows,data window>> for displaying evaluations and taps.
As soon as you evaluate or tap something it will be displayed here.
== Last evals
The last evals pane gives you access to the last 10 evaluation results, same as `*1` and `*2`.
Click on any value to display it on the top data window.
== Taps
Everytime _FlowStorm_ starts, it will add a tap, so whenever you `tap>` something
it will show on the taps list.
Click on any value to display it on the top data window.
If the tapped value has also been recorded as an expression in Flows, you can right click on it
and run `Search value on Flows` to move the debugger to that point in time.
[NOTE]
.Search value on Flows
====
Be aware that if the code that taps your value is something like `(tap> :a-key)` you won't be able to jump
to it using this, because `:a-key` isn't a value recorded by _FlowStorm_, while if the tapping
code is like `(tap> some-bind)` or `(tap> (+ 2 3))` or the tapping of any other expression
you should be able to jump to it.
So if you want to use this functionality as a "mark" so you can quickly jump to different parts of
the recordings from the Taps tool, you can do it like `(tap> (str :my-mark))`
====
A `#tap` tag will also be available, which will tap and return so you can use it like `(+ 1 2 #tap (* 3 4))`
Use the `clear` button to clear the list.
There is also `#tap-stack-trace`. It will tap the current stack trace.
== Out and Err streams
Everything written on `*out*` or `*err*` will be captured and displayed on the bottom panel.
You can copy anything from this area with normal tools.
= Data Windows
image::user_guide_images/data_window.png[]
Data Windows are a user extensible tool to visualize and explore your data. Their role is to support :
- a way to navigate nested structures in a lazy way
- visualize and navigate metadata
- multiple visualizations for each value
- lazy/infinite sequences navigation
- a way to define the current sub-values so you can use them at the repl
- a mechanism for realtime data visualization
- clojure.datafy navigation out of the box
- tools for the user to add custom visualizations on the fly
The next sections will explore each of them.
== Data navigation
image::user_guide_images/data_window_dig.png[]
You can navigate into any key or value by clicking on it.
Use the breadcrumbs at the top to navigate back.
== Metadata navigation
image::user_guide_images/data_window_meta.png[]
If any value contains metadata, it will be shown at the top. Clicking on it will make the data window
navigate into it.
== Multiple visualizers
image::user_guide_images/data_window_multiple_viz.png[]
You can change how to display your current value by using the visualizers selector dropdown at the top.
== Sequences
image::user_guide_images/data_window_seqable.png[]
The seqable visualizer allows you to navigate all kind of sequences (even infinite ones) by bringing more pages on demand.
Click on `More` to bring the next page in.
== Defining values
You can always define a var for the current value being shown on any data window by clicking the `def` button.
Clicking on it will raise a popup asking for a symbol name. If you don't provide a fully qualified symbol
it will define the var under `user` or `cljs.user` if you are in ClojureScript.
A quick way to use it is to provide a short name, let's say `foo`, and then access it from your
code like `user/foo`.
== Realtime visualizations
image::user_guide_images/data_window_realtime.png[]
DataWindows not only support displaying and navigating values, but also updating them in real time from
your application.
From your program's code you can always create a data window with :
[,clojure]
----
(flow-storm.api/data-window-push-val :changing-long-dw-id 0 "a-long")
----
by providing a data window id, a value, and optionally the initial breadcrumb label.
But you can also update it (given that the selected visualizer supports updating like :scope for numbers) with :
[,clojure]
----
(flow-storm.api/data-window-val-update :changing-long-dw-id 0.5)
----
This `data-window-val-update` is pretty useful when called from loops or refs watches, specially paired
with a custom visualizer.
== Clojure datafy/nav
image::user_guide_images/data_window_datafy_nav.png[]
Data Windows support datafy nav out of the box. The data window will always be showing the result of
`clojure.datafy/datafy` of a value. For maps or vectors where keys provide navigation it will automatically
add a blue arrow next to the value.
Clicking on the value will just dig the data, while clicking on the blue arrow will navigate as with
`clojure.datafy/nav` applied to that collection on that key.
== EQL pprint visualizer
image::user_guide_images/eql_visualizer_0.png[]
image::user_guide_images/eql_visualizer_1.png[]
The `eql-query-pprint` visualizer allows you to explore your data "entities" by looking at subsets of it
using queries similar to datomic pull queries like in the screenshots above.
[NOTE]
.Disable by default
====
The EQL query pprint is disable by default. To enable it call `(flow-storm.runtime.values/register-eql-query-pprint-extractor)`.
====
By entities it means maps which contains only keywords as their keys. Every other collection
is just traversed.
This are all valid queries :
- `[*]`
- `[:name]`
- `[:name :age :vehicles]`
- `[:name :age {:vehicles [:type]}]`
- `[:name :age {:vehicles [?]}]`
- `[:name {:vehicles [*]}]`
- `[:name :age {:vehicles [:type {:seats [?]}]}]`
- `[:name :age {:vehicles [:type {:seats [:kind]}]}]`
- `[:name {:houses [:rooms]}]`
The `*` symbol means include all keys, while the `?` symbol means just list the keys, which helps
exploring big nested maps with many keys.
== Custom visualizers
An important aspect of Data Windows is to be able to provide custom visualizers on the fly.
Let's say we have model a chess board as a set of maps which represent our pieces.
[,clojure]
----
(def chess-board
#{{:kind :king :player :white :pos [0 5]}
{:kind :rook :player :white :pos [5 1]}
{:kind :pawn :player :white :pos [5 3]}
{:kind :king :player :black :pos [7 2]}
{:kind :pawn :player :black :pos [6 6]}
{:kind :queen :player :black :pos [3 1]}})
(flow-storm.api/data-window-push-val :chess-board-dw chess-board "chess-board")
----
If we open a data window with `data-window-push-val` we are going to see something like this :
image::user_guide_images/data_window_custom1.png[]
but we can do better, we can create a custom visualizer so we can see it like this :
image::user_guide_images/data_window_custom2.png[]
Data visualization in FlowStorm is composed of two things:
- a data aspect extractor, which runs on the runtime process, and will build data for the visualization part
- a visualizer, which runs on the debugger process, and will render extracted data for a value using javafx
For a basic Clojure session everything will be running under the same process, but this is not the case for ClojureScript
or remote Clojure.
First let's require some namespaces :
[,clojure]
----
(require '[flow-storm.api :as fsa])
(require '[flow-storm.debugger.ui.data-windows.visualizers :as viz])
(require '[flow-storm.runtime.values :as fs-values])
----
We can register a custom visualizer by calling `register-visualizer`.
[,clojure]
----
(viz/register-visualizer
{:id :my-viz
:pred (fn [val] )
:on-create (fn [val] {:fx/node :any-java-fx-node-that-renders-the-value
:more-ctx-data :anything})
;; OPTIONALLY
:on-update (fn [val created-ctx-map {:keys [new-val]}] )
:on-destroy (fn [created-ctx-map] )
})
----
The important part there are `:id`, `:pred`, and `:on-create`.
The `:id` will be the one displayed on the visualizers dropdown, and re-registering a visualizer
with the same id will replace the previous one.
`:pred` is a predicate on the data extracted from values, it should return true if this visualizer
can handle the value.
And `:on-create` will be a function that receives this value and renders a java fx node.
The val passed to on-create will also contain two special keywords :
- :flow-storm.debugger.ui.data-windows.data-windows/dw-id The id of the data windows it's being draw on
- :flow-storm.debugger.ui.data-windows.data-windows/preferred-size (could be :small)
Optionally you can provide `:on-update` and `:on-destroy`.
`:on-update` will receive values from the runtime via `fsa/data-window-val-update`. It will also get a handle to
the original value (the one that created the DataWindow) and whatever map was returned by `:on-create`.
`:on-destroy` will be called everytime a visualizer gets removed, because you swapped your current visualizer
or because you went back with breadcrums. It can be useful in case you need to clear resources created by
`:on-create`.
`:pred` and `:on-create` will not receive the original value but the extracted aspects of it after
all registered extractors run.
You can check the data available to your visualizer for a value in a data window by calling :
[,clojure]
----
(viz/data-window-current-val :chess-board-dw)
----
If the data already extracted from your value is not enough for your visualizer you can register
another extractor.
=== Data aspect extraction
[,clojure]
----
(fs-values/register-data-aspect-extractor
{:id :chess-board
:pred (fn [val _]
(and (set? val)
(let [{:keys [kind player pos]} (first val)]
(and kind player pos))))
:extractor (fn [board _] {:chess/board board})})
----
In this case we are going to register and extractor that will only run for vals which are sets and
contains at least one element which is a map with `:kind`, `:player` and `:pos`.
The extracted data will be the entire board.
All ids of extractors that applied for a value will be appended under `::fs-values/kinds` of the value
as you will see next.
=== Visualizers
Now we can register a visualizer that will show only on values which contains a :chess-board kind.
[,clojure]
----
(import '[javafx.scene.layout GridPane])
(import '[javafx.scene.control Label])
(viz/register-visualizer
{:id :chess-board
;; only be available if the chess-board data extractor run on this value
:pred (fn [val] (contains? (::fs-values/kinds val) :chess-board))
;; use the chess/board info to render a board with java fx
:on-create (fn [{:keys [chess/board]}]
(let [kind->sprite {:king "♚" :queen "♛" :rook "♜" :bishop "♝" :knight "♞" :pawn "♟"}
pos->piece (->> board
(mapv #(vector (:pos %) %))
(into {}))]
{:fx/node (let [gp (GridPane.)]
(doall
(for [row (range 8) col (range 8)]
(let [cell-color (if (zero? (mod (+ col (mod row 2)) 2)) "#f0d9b5" "#b58863")
{:keys [kind player]} (pos->piece [row col])
cell-str (kind->sprite kind "")
player-color (when player (name player))]
(.add gp (doto (Label. cell-str)
(.setStyle (format "-fx-background-color:%s; -fx-font-size:40; -fx-text-fill:%s; -fx-alignment: center;"
cell-color player-color))
(.setPrefWidth 50))
(int col)
(int row)))))
gp)}))})
----
=== ClojureScript
Using custom visualizers with ClojureScript (or other remote environments) is a little bit more involved.
Registering aspect extractors is exaclty the same, since they run on the runtime (browswer, node, etc), but
custom visualizers should be registered on the debugger process. For this you need to create your visualizers in some
namespace, let's say on `/dev/visualizers.clj`, add the `dev` folder to your classpath and then running the debugger UI
with something like :
[,bash]
----
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-app :pre-require visualizers
----
Notice the last option `:pre-require visualizers`. This will allow you to load the just defined `visualizers` namespace before
starting the UI.
== Default visualizers
You can make any visualizer the default by calling `add-default-visualizer` which takes a predicate on the val-data (the same received by :on-create) and
a visualizer key, like this :
[,clojure]
----
(viz/add-default-visualizer (fn [val-data] (contains? (:flow-storm.runtime.values/kinds val-data) :chess-board)) :chess-board)
----
For all FlowStorm provided visualizers take a look at `flow-storm.debugger.ui.data-windows.visualizers` namespace.
Default visualizers predicates are added in a stack, and tried from the top. This means that you can always overwrite a default by adding a
new one.
= Thread breakpoints
image::user_guide_images/thread_breaks.png[]
_FlowStorm_ is a tracing debugger, which means it can record what is happening without the need of stopping
your programs execution. This is all fine but doesn't cover every possible situation. There are
cases where recording everything is impractical, and even if you can start/stop recording whenever you want,
being able to automatically stop your threads at certain points is useful.
For these cases, _FlowStorm_ has the ability to set thread breakpoints, which means to define points (functions)
in the execution of your program where you want your threads to wait.
While the threads are waiting you can explore what happened so far.
As soon as a thread hits a break function, if recording is on, it will be blocked, and a "Threads blocked" menu will show up in the UI.
You can use this menu to unblock different threads.
Once you are done, you can pause recording using the pause button in the main toolbar and un-block every thread.
You can define thread breakpoints in two ways :
- Using the browser (like in the image below), you can navigate to any function and click on the `Break` button. This will block the calling
thread every time the selected function gets called.
- Or you can also install a break by calling (flow-storm.api/break-at 'my-proj.core/some-fn)
image::user_guide_images/browser_breakpoints.png[]
[NOTE]
.Conditional threads breakpoints
====
The break-at fn accepts a second argument where you can provide a predicate that will be called with the same arguments
of the function you are breaking. It will only break when the predicate returns true. If you don't
provide a predicate it will default to `(constantly true)`
====
You can remove breakpoints by :
- Clicking on the browser instrumentation list delete buttons
- Calling `flow-storm.api/remove-break` to remove a single breakpoint
- Calling `flow-storm.api/clear-breaks` to remove all breakpoints
= Programmable debugging
_FlowStorm_ gives you full access to its internal indexes from the repl in Clojure and ClojureScript.
These allows you to explore your recordings using Clojure and write small programs to analyze
them if what's provided by the GUI is not enough.
Most of what is documented here is also documented in the `flow-storm.runtime.indexes.api` namespace docstring, which
you can retrieve by evaluating `(doc flow-storm.runtime.indexes.api)`. In fact, this is the only namespace you need to
require on your repl in order to work with your recordings.
Let's say you have recorded some execution and now you want to work with the recordings from the repl.
So first we require the api ns as `ia`.
[,clojure]
----
(require '[flow-storm.runtime.indexes.api :as ia])
----
Now from the UI, you can get the thread-id of your recordings (the number next to the thread name)
which you will need for accessing them from the repl.
== Timelines
Let's say you want to explore recordings on thread 32. You can retrieve its timeline by calling `ia/get-timeline` like this :
[,clojure]
----
(def timeline (ia/get-timeline 32))
----
Once you have the timeline you can start exploring it.
The timeline implements many of the Clojure basic interfaces, so you can :
[,clojure]
----
user> (count timeline)
798
user> (take 3 timeline)
; (#flow-storm/fn-call-trace [Idx: 0 org.my-app/run-server]
; #flow-storm/fn-call-trace [Idx: 1 org.my-app/read-config]
; #flow-storm/fn-call-trace [Idx: 2 org.my-app/check-config])
user> (get timeline 0)
; #flow-storm/fn-call-trace [Idx: 0 org.my-app/run-server]
----
The easiest way to take a look at a thread timeline is with some code like this :
[,clojure]
----
(->> timeline
(take 3)
(map ia/as-immutable))
; ({:type :fn-call,
; :fn-ns "org.my-app",
; :fn-name "run-server",
; :ret-idx 797,
; :fn-call-idx 0,
; :parent-idx nil,
; :fn-args [],
; :form-id -798068730,
; :idx 0}
; ...
; ...)
----
In most cases converting all entries into maps with `ia/as-immutable` is enough, but if you want a little bit more
performance you can access entries information without creating a immutable map first.
Timelines entries are of 4 different kinds: `FnCallTrace`, `FnReturnTrace`, `FnUnwindTrace` and `ExprTrace`.
You can access their data by using the following functions depending on the entry :
All kinds :
- `as-immutable`
- `fn-call-idx`
`ExprTrace`, `FnReturnTrace` and `FnUnwindTrace` :
- `get-coord-vec`
`ExprTrace`, `FnReturnTrace` :
- `get-expr-val`
`FnUnwindTrace` :
- `get-throwable`
`FnCallTrace` :
- `get-fn-name`
- `get-fn-ns`
- `get-fn-args`
- `get-fn-parent-idx`
- `get-fn-ret-idx`
- `get-fn-bindings`
You can also access the timeline as a tree by calling :
- `callstack-root-node`
- `callstack-node-childs`
- `callstack-node-frame-data`
Take a look at their docstrings for more info.
== Forms
You can retrieve forms by form id with `get-form` and then use `get-sub-form-at-coord` and a coordinate.
Here is a little example :
[%nowrap,clojure]
----
;; retrieve some expression entry into expr
user> (def expr (-> timeline
(get 3)
ia/as-immutable))
user> expr
{:type :expr, :coord [2 2 1], :result 4, :fn-call-idx 2, :idx 3}
;; retrieve the fn-call entry for our expr
user> (def fn-call (-> timeline
(get (:fn-call-idx expr))
ia/as-immutable))
user> fn-call
{:type :fn-call,
:fn-ns "dev-tester"
:fn-name "other-function",
:form-id 1451539897,
...}
;; grab it's form
user> (def form (-> fn-call
:form-id
ia/get-form
:form/form))
user> form
(def other-function (fn [a b] (+ a b 10)))
;; lets look at the sub-form from form at our expr coordinate
user> (ia/get-sub-form-at-coord form (:coord expr))
a
----
== Multi-thread timeline
If you have recorded a multi-thread timeline, you can retrieve it with `total-order-timeline` like this :
[,clojure]
----
(def mt-timeline (ia/total-order-timeline))
----
which you can then iterate using normal Clojure functions (map, filter, reduce, get, etc).
The easiest way to explore it is again with some code like this :
[,clojure]
----
user> (->> mt-timeline
(take 3)
(map ia/as-immutable))
({:thread-id 32,
:type :fn-call,
:fn-call-idx 0,
:fn-ns "org.my-app",
:fn-name "run",
:fn-args [],
:ret-idx 797,
:parent-idx nil,
:form-id -798068730,
:idx 0}
...
...)
----
Notice that each of these entries contains a flow-id and thread-id also.
== Other utilities
There are other utitities in the api ns that could be useful, some of the most interesting ones :
- `find-expr-entry` useful for searching expressions and return values with different criteria.
- `find-fn-call-entry` useful for searching functions calls with different criteria.
- `stack-for-frame`
- `fn-call-stats`
Take a look at their docstrings for more info.
= LLM agents
You can teach a LLM how to use FlowStorm's api to help you analyze your recordings.
If you are using the amazing https://github.com/bhauman/clojure-mcp[clojure-mcp] you just need to upload
https://github.com/flow-storm/flow-storm-debugger/blob/master/llm-prompt.txt[one more file] that teaches the LLM FlowStorm's basics from the repl.
https://claude.ai/share/489c9124-b1a8-4a33-b50a-52e4f3d4709f[Here] is a very basic chat asking Claude to look at some recordings of a buggy
TODO's web application.
= Remote debugging
You can remotely debug any Clojure application that exposes a nrepl server.
In terms of dependencies, the debuggee side should be setup the same as a normal local setup, with the optional change that you can use
`flow-storm-inst` instead of `flow-storm-dbg`, being the former a slimmed down version of the later one that
doesn't contain some libraries used only by the UI, but using the full `flow-storm-dbg` is also ok.
== SSH tunnel
The easiest way to debug a remote application is via a ssh tunnel. You can create it from your dev box like this :
[,bash]
----
ssh -L 9000:localhost:9000 -R 7722:localhost:7722 my-debuggee-box.com
----
assuming your remote process at my-debuggee-box.com has started a nrepl server listening on port 9000 and
that the debugger websocket server is running on the default port.
After the tunnel is established, you can run you debugger UI like this :
[,bash]
----
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger :port 9000
----
and that is it.
If you need to connect the debugger to a remote process without a ssh tunnel or you need to configure the websocket server port you can do it like this :
[,bash]
----
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger :port NREPL-PORT :runtime-host '"YOUR-APP-BOX-IP-ADDRESS"' :debugger-host '"YOUR-BOX-IP-ADDRESS"' :ws-port WS-SERVER-PORT
----
== Out of process
Sometimes you are not debugging across a network but you want to run the FlowStorm UI on a different process.
A couple of aliases that can help for this :
[,clojure]
----
{:aliases
;; for your system process
{:runtime-storm {:classpath-overrides {org.clojure/clojure nil}
:extra-deps {com.github.flow-storm/clojure {:mvn/version "1.12.4"}
com.github.flow-storm/flow-storm-inst {:mvn/version "4.5.9"}}}
;; for the FlowStorm GUI process
:ui-storm {:extra-deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}
:exec-fn flow-storm.debugger.main/start-debugger
:exec-args {:port 7888}}}} ;; set your nrepl port here!
----
With those aliases you can start your application process by adding `:runtime-storm` and that is it.
To start the FlowStorm UI, go to another terminal and run `clj -X:ui-storm`.
== Docker
If you run you process inside a docker container, here is a basic template for using _FlowStorm_ with it
https://github.com/jpmonettas/docker-flow-storm-basic
= Dealing with too many traces
When recording an application's execution (specially when using _ClojureStorm_ or _ClojureScriptStorm_) it could happen
that your process starts running out of heap. This section documents some tools FlowStorm provides to deal with this situations.
Most of the time, having the recording paused and just enabling it right before executing the action you are interested in is enough, but
when it isn't, here are some other options.
== Fn call limits
A common situation is to see some high frequency functions adding a lot of noise to your recordings. For example a MouseMove event processing
will generate a lot of recordings while you use your app. There are a couple of ways to limit your functions calls by thread. You can identify
this kind of functions with the <<#_functions_tool,functions tool>>.
One tool you can use in this situations is the `flowstorm.threadFnCallLimits` JVM prop.
For example, you can add `"-Dflowstorm.threadFnCallLimits=org.my-app/fn1:2,org.my-app/fn2:4"` so every time the system starts, limits will be set for
`org.my-app/fn1` and `org.my-app/fn2`. The number next to them is the limit. When a function reaches the limit _FlowStorm_ will stop recording calls
to it and all the functions down its callstack.
You can also modify the limits from your repl, by calling `flow-storm.runtime.indexes.api/[add-fn-call-limit|rm-fn-call-limit|get-fn-call-limits]`.
In ClojureScript you need to call them via your cljs repl api.
These limits are per thread, so when a thread recording is created it will start with the current defined counters, and each time a function gets called
the counter will decrement. When it reaches zero the function and all functions calls under it will stop being recorded.
When you clear your threads you are also clearing its limit counters, so next time you record something new counters will be initialized from your
global limits definitions.
== Trace and heap limits
If you are tracing some code that ends up in a infinite loop the debugger will choke on
too many traces, making everything slow and your only option is probably to restart it.
For preventing this, _FlowStorm_ provides a couple of fuse/breakers, on threads trace count and on heap limits.
They are off by default but you can enable it from the Config menu.
=== Trace limits
Let's say you added a thread trace limit of 1000. If you now run any code where a thread generates more than a 1000 traces FlowStorm will only
record those first 1000 traces and then discard the rest as if recording is off for that thread, while it will keep recording threads that haven't
reached the limit.
Your code will continue execution as normal, which you can break using
your normal editor breaking commands if its an infinite loop, but now you have recordings to look at what is going on.
You can set a limit of 0 to disable it again.
You can set this limits at startup via the JVM options `"-Dflowstorm.threadTraceLimit=1000"` and
`"-Dflowstorm.throwOnLimit=true"`.
=== Heap limits
Another option is to automatically stop recording when a certain heap limit in megabytes is reached.
This can also be set at startup via the JVM option `"-Dflowstorm.heapLimit=1000"`, which means stop recording
as soon as we used 1000Mb of heap space.
= Dealing with mutable values
_FlowStorm_ will retain all values pointers when code executes so you can analyze them later. This works great with immutable values but
when your code uses mutable values like this :
[,clojure]
----
(let [a (java.util.ArrayList.)]
(count a)
(.add a "hello")
(count a)
(.add a "world")
(.add a "!"))
----
then every time you step over `a` it will contain the last value ["hello" "world" "!"].
You can fix this situation by extending the flow-storm.runtime.values/SnapshotP protocol like this :
[,clojure]
----
(extend-protocol flow-storm.runtime.values/SnapshotP
java.util.ArrayList
(snapshot-value [a] (into [] a)))
----
to provide _FlowStorm_ a way of creating a snapshot of the mutable value.
[NOTE]
.ClojureStorm
====
If you are using _ClojureStorm_ evaluate the previous defmethod in a ns that is not being
instrumented to avoid an infinite recursion.
====
Be aware that this is tricky in multithreading situations, as always with mutable values.
[NOTE]
.Automatic derefing
====
FlowStorm will automatically deref Atoms, Refs, Agents, Vars and all pending-realized derefables on tracing
so no need to implement `flow-storm.runtime.values/snapshot-value` for them.
====
[NOTE]
.Snapshoting and nested values
====
Snapshoting only applies to direct references to mutable values.
For example if you have an atom inside a nested immutable collection, it will not be snapshoted every time
that collection expression is being recorded, because the value being recorded is not a reference to an atom.
If this is important to you, you can still define snapshot-value for clojure.lang.PersistentArrayMap, etc, and
walk it down snapshoting everything mutable inside.
====
[NOTE]
.snapshot-value and memory footprint
====
Although snapshot-value was created as a way to deal with mutable values it can be used to replace any value by another in the
recordings, which can be useful in other situations like reducing memory footprint when you don't need the entire
value to be recorded.
====
= Controlling instrumentation
If you are using _ClojureStorm_ or _ClojureScriptStorm_ it is important to learn how to control what gets instrumented and
how to uninstrument things. You can configure what gets instrumented automatically on startup via JVM properties but also
change this while your repl is running without the need to restart it.
FlowStorm by default will automatically figure out what to instrument from your project, which
you can always disable by setting the `-Dclojure.storm.instrumentAutoPrefixes=false`.
[NOTE]
.How are auto prefixes calculated?
====
When the process starts it will scan all source folders on the classpath (everything not inside a jar containing clojure files) and build
a set of all top level namespace. All namespaces under those will be instrumented.
Currently it doesn't detect single level namespaces, like when you have `src/core.clj`, if this is your case use
instrumentOnlyPrefixes.
====
If you prefer to be explicit about what gets instrumented you can use the JVM property `"-Dclojure.storm.instrumentOnlyPrefixes=YOUR_INSTRUMENTATION_STRING"`
where `YOUR_INSTRUMENTATION_STRING` should be a comma separated list of namespaces prefixes like :
my-project.,lib1.,lib2.core
which means automatically instrument my-project.* (which includes all sub namespaces), all lib1.* and only everything under lib2.core
All this can be changed after without restarting your repl from <<#_modifying_instrumentation_with_the_browser, FlowStorm browser>>.
== Turning instrumentation on an off
You can turn instrumentation on an off by using the button on <<#_the_tool_bar,the toolbar>>.
Remember that the change of this setting will only be effective on newly compiled code.
== Setup startup instrumentation
The first important thing is to setup your instrumentation correctly via JVM properties :
On _ClojureStorm_ :
[,clojure]
----
-Dclojure.storm.instrumentOnlyPrefixes=my-app,my-lib
-Dclojure.storm.instrumentSkipPrefixes=my-app.too-heavy,my-lib.uninteresting
-Dclojure.storm.instrumentSkipRegex=.*test.*
----
On _ClojureScriptStorm_ :
[,clojure]
----
-Dcljs.storm.instrumentOnlyPrefixes=my-app,my-lib
-Dcljs.storm.instrumentSkipPrefixes=my-app.too-heavy,my-lib.uninteresting
----
Apart from `instrumentOnlyPrefixes` which you probably already know, there is `instrumentSkipPrefixes` which also
accepts a comma separated list of namespaces prefixes to skip, and instrumentSkipRegex with accepts a regex for
namespaces to skip. All these together allows you to instrument you whole app but some undesired namespaces.
The next important thing is to be able to enable/disable instrumentation and add/remove prefixes without restarting the
repl.
== Modifying instrumentation with the Browser
You can use the `Browser tool` to check and change on the fly the prefixes you configured in the previous section.
image::user_guide_images/browser_storm_instrumentation_1.png[]
Right clicking any namespace will give you options for what level of a namespace you want to instrument.
On the bottom pane (instrumentations) you will see your current instrumentation configuration (if any). Here we can see that everything under
`ring.middleware.anti-forgery` will be instrumented every time something inside it gets compiled.
You can remove entries using the `del` buttons or temporarily disable/enable them using the `Enable all` checkbox.
image::user_guide_images/browser_storm_instrumentation_2.png[]
You can use the `Add` menu in the picure above to add instrumentation prefixes.
After changing any prefix FlowStorm will ask if you want it to reload the affected namespaces for you. Namespace reloading will
all reload all namespaces it depends on in topological order, so it shouldn't break your system in any way.
image::user_guide_images/browser_storm_instrumentation_3.png[]
You can also provide functions to be called before and after reloading in case you need to stop and start your system with :
[,clojure]
----
(flow-storm.api/set-before-reload-callback! (fn [] (println "Before reloading")))
(flow-storm.api/set-after-reload-callback! (fn [] (println "After reloading")))
----
[NOTE]
.Instrumentation
====
Just changing the prefixes without reloading will not make your currently loaded code [un]instrumented.
If you haven't let FlowStorm reload them for you, you can always recompile them as usual with your editor commands or
by executing something like `(require 'the-selected.namespace :reload)`.
====
== Instrumentation in Vanilla FlowStorm
[NOTE]
.ClojureStorm
====
Instructions here only apply to vanilla _FlowStorm_. If you are using _ClojureStorm_ or _ClojureScriptStorm_ (recommended)
this is done automatically for you, so just skip this section.
====
Code instrumentation in _FlowStorm_ is done by rewriting your code, in a way that doesn't change its behavior
but when executed will trace everything the code is doing.
=== Instrument any form with #trace
You can instrument any top level form at the repl by writing `#trace` before it, like this :
[,clojure]
----
#trace
(defn sum [a b]
(+ a b))
----
and then evaluating the form.
important:: `#trace` is meant to be used with forms that don't run immediately, like: defn, defmethod, extend-type, etc.
Use `#rtrace` to trace and run a form, like `#rtrace (map inc (range 10))`.
=== Run code with #rtrace
`#rtrace` is useful in two situations :
First, when instrumenting and running a simple form at the repl, like:
[,clojure]
----
#rtrace (-> (range) (filter odd?) (take 10) (reduce +))
----
=== Instrument namespaces
_FlowStorm_ allows you to instrument entire namespaces by providing `flow-storm.api/instrument-namespaces-clj`.
You call it like this :
[,clojure]
----
(instrument-namespaces-clj #{"org.my-app.core" "cljs."})
----
The first argument is a set of namespaces prefixes to instrument. In the previous example it means
instrument all namespaces starting with `org.my-app.core`, and all starting with `cljs.`
The second argument can be a map supporting the following options :
- `:excluding-ns` a set of strings with namespaces that should be excluded
- `:disable` a set containing any of #{`:expr` `:binding` `:anonymous-fn`} useful for disabling unnecessary traces in code that generate too many
- `:verbose?` when true show more logging
=== What can't be instrumented?
These are some limitations when instrumenting forms :
1. Very big forms can't be fully instrumented. The JVM spec has a limit on the size of methods and instrumentation adds a lot of code.
When instrumenting entire namespaces, if you hit this limit on a form a warning will printed on the console saying `Instrumented expression is too large for the Clojure compiler`
and _FlowStorm_ automatically tries to instrument it with a lighter profile, by disabling some instrumentation.
2. Functions that call recur without a loop
3. Functions that return recursive lazy sequences. Like `(fn foo [] (lazy-seq (... (foo))))`
=== Un-instrument code
Un-instrumenting code that has been instrumented with `#trace` or `#ctrace` is straight forward, just remove the tag and re evaluate the form.
To un-instrument entire namespaces you can use `flow-storm.api/uninstrument-namespaces-clj` which accept a set of namespaces prefixes.
=== Instrument with the browser
Most of the time you can instrument code by just clicking in the browser. The only exceptions are functions that were just defined in the repl
and weren't loaded from a file.
==== Instrument vars
Using the browser you can navigate to the var you are interested in and then use the instrument button to instrument it.
image::user_guide_images/browser_var_instrumentation.png[]
There are two ways of instrumenting a var :
- Instrument (instrument just the var source code)
- Instrument recursively (recursively instrument the var and all vars referred by it)
==== Instrument namespaces
Using the browser you can also instrument multiple namespaces. Do this by selecting the namespaces you are interested in
and then a right click should show you a menu with two instrumentation commands.
image::user_guide_images/browser_ns_instrumentation.png[]
- `Instrument namespace :light` - record function arguments and return values (not expressions, no bindings tracing)
- `Instrument namespace :full` fully instrument everything
Light instrumentation is useful when you know the functions generate too many traces, so you can opt to trace just functions
calls and returns. You can then <<#_fully_instrument_a_form_from_the_code_view, fully instrument>> whatever functions you are interested in.
==== Un-instrument code
The bottom panel shows all instrumented vars and namespaces.
image::user_guide_images/browser_uninstrument.png[]
You can un-instrument them temporarily with the enable/disable checkbox or
permanently with the del button.
==== Fully instrument a form from the code view
image::user_guide_images/fully_instrument_form.png[]
If you have instrumented a form with the <<#_instrument_namespaces_2, :light profile>> you can fully instrument it by right clicking on the current form
and then clicking `Fully instrument this form`.
= Plugins
FlowStorm plugins allows you to add specialized tools to visualize and interact
with your recordings.
== Using plugins
For using a pluggin follow each plugging instructions which should normally consists of adding
its dependency and then setting the jvm prop `flowstorm.plugins.namespaces` with all the main
namespaces of the plugins you want loaded at startup, like : `"-Dflowstorm.plugins.namespaces=flow-storm.plugins.my-plugin.all"`
After that you should see a new vertical tab with the plugin UI as you can see here :
image::user_guide_images/plugin_demo.png[]
== Creating plugins
Creating a pluging consists of two parts :
- The runtime code that will analyze the recordings and expose an api for the UI
- The UI component which will visualize and interact with the data via the runtime api
This split is not required, but it is important if you want your plugin to support ClojureScript also or
remote Clojure debugging where the UI is not running in the same process as the runtime.
This components are normally split in two files, a runtime.clj and ui.clj, but you can name them however you
want.
We are going to go over each part in more detail but for a real plugin please checkout the https://github.com/flow-storm/flow-storm-async-flow-plugin[core.async.flow plugin].
=== Runtime
Here is a runtime file template you can use :
[%nowrap,clojure]
----
(ns flow-storm.plugins.my-plugin.runtime
(:require [flow-storm.runtime.indexes.api :as ia]
[flow-storm.runtime.debuggers-api :as dbg-api]))
(defn my-data-extraction [flow-id thread-id]
(let [timeline (ia/get-timeline flow-id thread-id)]
(reduce (fn [acc tl-entry]
;; extract some interesting info from the timeline
)
{}
timeline)
))
;; Expose your function so it can be called from the UI part
(dbg-api/register-api-function :plugins.my-plugin/extract-data my-data-extraction)
----
=== UI
Here is a ui file template you can use :
[%nowrap,clojure]
----
(ns flow-storm.plugins.my-plugin.ui
(:require [flow-storm.debugger.ui.plugins :as fs-plugins]
[flow-storm.debugger.runtime-api :as runtime-api :refer [rt-api]])
(:import [javafx.scene.control Label]))
(fs-plugins/register-plugin
:my-plugin
{:label "My plugin"
:css-resource "flow-storm-my-plugin/dark.css"
:dark-css-resource "flow-storm-my-plugin/dark.css"
:light-css-resource "flow-storm-my-plugin/light.css"
:on-focus (fn [{:keys [some-other-data]}]
;; This gets called everytime the plugin tab gets focused
)
:on-create (fn [_]
{:fx/node (Label.
;; You can call your runtime registered function
(str (runtime-api/call-by-fn-key rt-api :plugins.my-plugin/extract-data [0 10])))
:some-other-data 42})
:on-flow-clear (fn [flow-id {:keys [some-other-data]}]
;; this gets called everytime a flow is discarded so you can update your plugin UI accordignly
)
})
----
==== Styling plugins UIs
As you saw in the ui plugin registration, you can provide three resources related to styling :
- :css-resource If there is any, it will be loaded and applied. Here is where you put your JavaFX pluging styles
- :dark-css-resource This styles are going to be applied only in dark mode
- :light-css-resource This styles are going to be applied only in light mode
For making sure you plugin styles doesn't mix with other styles, your plugin is automatically wrapped in a pane
with your plugin key (my-plugin in the example above) as a class.
This means your plugin css can contain code like :
[%nowrap,clojure]
----
.my-plugin .table-view {
-fx-font-family: 'monospaced';
}
----
== List of known plugins
- https://github.com/flow-storm/flow-storm-web-plugin
- https://github.com/flow-storm/flow-storm-flowbook-plugin
- https://github.com/flow-storm/flow-storm-cljs-compiler-plugin
- https://github.com/flow-storm/flow-storm-async-flow-plugin
= JVM options list
This section only collects the options, search for them in the User's guide for more context
and possible values.
== Clojure and ClojureScript
- `-Dflowstorm.startRecording=false`
- `-Dflowstorm.plugins.namespaces[.+]=ns1,ns2`
- `-Dflowstorm.threadFnCallLimits=org.my-app/fn1:2,org.my-app/fn2:4`
- `-Dflowstorm.title=FlowStormMainDebugger`
- `-Dflowstorm.theme=dark`
- `-Dflowstorm.styles=~/.flow-storm/big-fonts.css`
- `-Dflowstorm.fileEditorCommand=emacsclient -n +\<>:0 \<>`
- `-Dflowstorm.jarEditorCommand=emacsclient -n +\<>:0 \<>/\<>`
- `-Dflowstorm.threadTraceLimit=1000`
- `-Dflowstorm.throwOnLimit=true`
- `-Dflowstorm.autoUpdateUI=false`
- `-Dflowstorm.callTreeUpdate=false`
- `-Dflowstorm.uiTimeoutMillis=4000`
== Only Clojure
- `-Dclojure.storm.instrumentEnable=true`
- `-Dclojure.storm.instrumentOnlyPrefixes[.*]=ns-prefix1,ns-prefix2`
- `-Dclojure.storm.instrumentAutoPrefixes=false`
- `-Dclojure.storm.instrumentSkipPrefixes[.*]=my-app.too-heavy,my-lib.uninteresting`
- `-Dclojure.storm.instrumentSkipRegex=.\*test.*`
- `-Dflowstorm.heapLimit=1000`
== Only ClojureScript
- `-Dcljs.storm.instrumentEnable=true`
- `-Dcljs.storm.instrumentOnlyPrefixes=ns-prefix1,ns-prefix2`
- `-Dcljs.storm.instrumentAutoPrefixes=false`
- `-Dcljs.storm.instrumentOnlyPrefixes=my-app,my-lib`
- `-Dcljs.storm.instrumentSkipPrefixes=my-app.too-heavy,my-lib.uninteresting`
= Styling and theming
All functions that start the debugger ui (`flow-storm.api/local-connect`, `flow-storm.debugger.main/start-debugger`) accept a map
with the `:styles`, `:title` and `:theme` keywords. If `:styles` points to a css file it will be used to overwrite the default styles, in case you
want to change colors, make your fonts bigger, etc. `:theme` could be one of `:auto` (default), `:light`, `:dark`. Title can be used to
distinguish between multiple debugger instances.
Like this :
[,clojure]
----
user> (local-connect {:styles "~/.flow-storm/big-fonts.css", :theme :dark, :title "FlowStormMainDebugger"})
----
If you are using _ClojureStorm_ you can also provide them with :
-Dflowstorm.title=FlowStormMainDebugger
-Dflowstorm.theme=dark
-Dflowstorm.styles=~/.flow-storm/big-fonts.css
You can overwrite all the styles defined here https://github.com/flow-storm/flow-storm-debugger/blob/master/resources/flowstorm/styles/styles.css
= Controlling logging
FlowStorm uses JUL (java.util.logging) as the loggging library.
You can configure JUL logging by starting your repl with `-Djava.util.logging.config.file=./logging.properties`
If you need to disable logging you can put this in your `logging.properties` file :
[,text]
----
handlers = java.util.logging.ConsoleHandler
flow_storm.level = SEVERE
clojure.storm.level = SEVERE
----
= Key bindings
== General
- `Ctrl-g` Cancel any long running task (only search supported yet)
- `Ctrl-l` Clean all debugger state
- `Ctrl-d` Toggle debug-mode. Will log useful debugging information to the console
- `Ctrl-u` Unblock all breakpoint blocked threads if any
- `Ctrl-t` Rotate themes
- `Ctrl-plus` Increment font size
- `Ctrl-minus` Decrement font size
- `F` "Select the Flows tool"
- `B` "Select the Browser tool"
- `T` "Select the Taps tool"
- `D` "Select the Docs tool"
== Flows
- `0-9` Open focus flow-N threads menu, N being the pressed key
- `t` Select the tree tool (needs to be inside a thread)
- `c` Select the code tool (needs to be inside a thread)
- `f` Select the functions tool (needs to be inside a thread)
- `P` Step prev over. Go to previous step on the same frame
- `p` Step prev
- `n` Step next
- `N` Step next over. Go to next step on the same frame
- `^` Step out
- `<` Step first
- `>` Step last
- `Ctrl-f` Copy current function symbol
- `Ctrl-Shift-f` Copy current function call form
- `Ctrl-z` Undo navigation
- `Ctrl-r` Redo navigation
= Debugging react native applications
Debugging ClojureScript react native application needs a combination of ClojureScript and remote debugging.
Assuming you are using shadow-cljs, have added the `flow-storm-inst` dependency, and that it started a nrepl server on port 9000, you
can start a debugger and connect to it by running :
[,bash]
----
clj -Sforce -Sdeps '{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}' -X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :your-app-build-id :debugger-host '"YOUR_DEV_MACHINE_IP"'
----
You also need to make it possible for the device to connect back to the debugger on port 7722. You can accomplish this by running :
[,bash]
----
adb reverse tcp:7722 tcp:7722
----
Also remember that you need to have installed the `websocket` npm library. You can do this like :
[,bash]
----
npm install websocket --save
----
= Working on Windows with WSL2
For those using current versions of WSL2 on Windows it should be pretty straight forward.
- export DISPLAY=:0
- export WSL2_GUI_APPS_ENABLED=1
Font issues had been reported on some distros, like `java.lang.NullPointerException: Cannot read field "firstFont" because "" is null` which
seams to be solved just by installing font packages like `dejavu-fonts` or `ttf-dejavu` depending on the distro.
= Opening forms in editors
You can add this two jvm options to tell FlowStorm how to open forms in files and inside jars :
- flowstorm.jarEditorCommand : a command with optional \<>, \<> and \<> placeholders
- flowstorm.fileEditorCommand : a command with optional \<> and \<> placeholders
If you define those, clicking on your forms namespaces link in the code tool should run the provided commands.
On expressions sub-forms that contains line meta you should also be able to right click and select "Open in editor"
which should open the file at that specific line (useful for long forms).
Here are some known setups for most common editors :
== Emacs
[,clojure]
----
;; for opening your project files
"-Dflowstorm.fileEditorCommand=emacsclient -n +<>:0 <>"
;; simple way for opening files inside jars (works on linux only)
"-Dflowstorm.jarEditorCommand=emacsclient -n +<>:0 <>/<>"
;; for opening files inside jars that works on every OS (requires FlowStorm >= 3.17.3)
"-Dflowstorm.jarEditorCommand=emacsclient --eval '(let ((b (cider-find-file \"jar:file:<>!/<>\"))) (with-current-buffer b (switch-to-buffer b) (goto-char (point-min)) (forward-line (1- <>))))'"
----
== VSCode
[,clojure]
----
"-Dflowstorm.fileEditorCommand=code --goto <>:<>"
----
== IntelliJ
[,clojure]
----
"-Dflowstorm.fileEditorCommand=idea --line <> <>"
----
== Vim
[,clojure]
----
"-Dflowstorm.fileEditorCommand=vim +<> <>"
----
= Editors/IDEs integration
== Emacs
Checkout https://github.com/flow-storm/cider-storm[Cider Storm] an Emacs Cider front-end with support for Clojure and ClojureScript.
== VSCode
With the following alias setup in deps.edn:
[source,clojure]
{:aliases {:flowstorm {:classpath-overrides {org.clojure/clojure nil}
:extra-deps {com.github.flow-storm/clojure {:mvn/version "1.12.4"}
com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}
:jvm-opts ["-Dflowstorm.startRecording=true"
"-Dclojure.storm.intrumentEnable=true"
"-Dclojure.storm.intrumentAutoPrefixes=true"]}}}
Create a custom connect sequence in the VSCode settings.json:
[source,json]
{
"name": "flowstorm",
"projectType": "deps.edn",
"cljsType": "none",
"extraNReplMiddleware": ["flow-storm.nrepl.middleware/wrap-flow-storm"],
"afterCLJReplJackInCode": "((requiring-resolve 'flow-storm.storm-api/start-debugger))",
"menuSelections": {
"cljAliases": ["flowstorm"]
}
}
Jack-in using the `flowstorm` sequence from the menu.
== IntelliJ IDEA (Cursive)
= Tutorials and demos
https://github.com/flow-storm/flow-storm-debugger?tab=readme-ov-file#some-demo-videos-newers-at-the-top
= Known limitations and issues
== Code with macros that don't preserve meta
FlowStorm works fine with most macros, except the ones that don't preserve meta at macroexpansion, which FlowStorm needs in their absence,
can cause GAPS IN EXECUTION TRACING AND PREVENT IT FROM LINKING THE EXECUTED CODE BACK TO THE ORIGINAL SOURCE.
When macros are involved, the forms compiled by the Clojure compiler aren't the ones on your source files, but the ones generated
by macro-expanding all the macros. In order to link the compiled forms back to the forms in your source code, for each instrumented form,
right after the form is read by the reader, FlowStorm will walk the form down, annotating with meta each sub-form with a coordinate,
which will then be used after macro expansion to link a compiled expression back to your source code.
Macros can be as simple as code reorganizing ones (like `->`, `when`, `and`, `defn`, etc) or whole compilers like `clojure.core.async/go` and
Electric, so it really depends on the macros.
If you see code inside a macro not being traced feel free to report an issue, there is nothing FlowStorm can do from its side but
we can work together with the macro developer making sure it preserves all meta after macro expansion, which sometimes may be possible.
== Locals pane and mutable values
The Locals pane will show the value of each binding for a symbol AT BINDING TIME, which is the same thing
no matter where you are in the current block when working with immutable objects, BUT NOT WHEN WORKING WITH MUTABLE ONES.
If what was bound was mutable in any way, you will be seeing the value at binding time, and not at current time which
could cause some confusion.
== Closed over bindings aren't displayed in Locals
The locals pane will only display bindings for the current function. Locals visible from the current function but not defined
in it (like in the case of closures) aren't shown.
== Don't instrument clojure.core or FlowStorm itself
Currently we can't instrument clojure.core or FlowStorm itself since they endup in infinite tracing recursions.
This can be solved, but it is currently a limitation.
== IF test expressions tracing with intrinsics
When you have code like :
[,clojure]
----
(defn foo [^long l]
(if (zero? l)
(+ l 1)
(+ l 2)))
----
because `l` is a primitive long, the compiler can replace the (zero? l) with intrinsics (LCONST_0, LCMP, IFNE) so the
(zero? l) isn't a expression anymore, just a statement. In these cases you will see the if test return un-highlighted, but
you can still tell which branch the code went thru because the chosen branch will be the highlighted one.
== Loading an entire file while recording records some weird code
This is because most editors, (specially via nrepl) eval some other code on your namespace in order to load the
contents of your file. If that namespace is instrumented this will be also recorded, even when it is probably not
of your interest.
This is harmless, just clear your recordings before running and recording anything else.
If you follow the best practice of start recording right before running the stuff you are interested in recording you
should never see this.
== Macro calls tracing code you don't see on your code
When you are evaluating some code that macroexpands to a (do form-1 ... form-n) the compiler recursively calls eval on the
sub forms. Because it is tricky in the compiler to tell apart your original source form from the ones the macroexpansion returned
those form-1 to form-n get instrumented and then traced as if they were on your code.
The tricky part is related to tooling like IDEs sometimes wrapping your forms in some macros that expand to a (do form-1 ... form-n)
so we can't simply stop instrumenting after that situation.
= Troubleshooting
== The outputs panel doesn't work
Checkout that you don't have piggieback on the classpath dragged by some dependency. Currently
if piggieback is pressent FlowStorm will assume a ClojureScript repl in which the outputs panel
isn't supported yet.
== Run with JDK 11
FlowStorm UI requires JDK >= 17. If you can't upgrade your JDK you can still use it by downgrading JavaFx.
If that is the case add these dependencies to your alias :
[,clojure]
----
org.openjfx/javafx-controls {:mvn/version "19.0.2"}
org.openjfx/javafx-base {:mvn/version "19.0.2"}
org.openjfx/javafx-graphics {:mvn/version "19.0.2"}
org.openjfx/javafx-web {:mvn/version "19.0.2"}
----
= Internals, diagrams and documentation
For people interested in enhancing, troubleshooting, fixing or just learning about FlowStorm internals take a look at here :
https://github.com/flow-storm/flow-storm-debugger/blob/master/docs/dev_notes.md
Some useful diagrams :
- https://raw.githubusercontent.com/flow-storm/flow-storm-debugger/master/docs/high_level_diagram.svg
- https://raw.githubusercontent.com/flow-storm/flow-storm-debugger/master/docs/timeline.svg
- https://raw.githubusercontent.com/flow-storm/flow-storm-debugger/master/docs/run_configs.svg
////
Local Variables:
mode: outline
outline-regexp: "[=]+"
End:
////
================================================
FILE: docs/user_guide.html
================================================
FlowStorm debugger User’s Guide
This is the newest and simplest way of using FlowStorm. It requires you to swap your official Clojure compiler by ClojureStorm only at dev time.
Swapping compilers sounds like a lot, but don’t worry, ClojureStorm is just a patch applied over the official compiler with some
extra stuff for automatic instrumentation. You shouldn’t encounter any differences, it is only for dev, and you can swap it back
and forth by starting your repl with a different alias or lein profile.
The easiest way to run and learn FlowStorm with ClojureStorm is by running the repl tutorial.
You can start a repl with FlowStorm with a single command like this :
;; on Linux and OSX
clj -Sforce-Sdeps'{:deps {} :aliases {:dev {:classpath-overrides {org.clojure/clojure nil} :extra-deps {com.github.flow-storm/clojure {:mvn/version "1.12.4"} com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}}}'-A:dev
;; on Windows
clj -Sforce-Sdeps'{:deps {} :aliases {:dev {:classpath-overrides {org.clojure/clojure nil} :extra-deps {com.github.flow-storm/clojure {:mvn/version """1.12.4"""} com.github.flow-storm/flow-storm-dbg {:mvn/version """4.5.9"""}}}}}'-A:dev
Pasting that command on your terminal will bring up a repl with FlowStorm and the compiler swapped by ClojureStorm. When the repl comes up
evaluate the :dbg keyword to bring up the UI and then click on Help→Tutorial on the menu for a tour of the basics.
After the tutorial you may want to use it on your projects. You use it by adding a deps.edn alias or lein profile.
The simplest way is to setup it globally, so that is what we are going to do next. You can also add it only
to specific projects if they require special configurations.
You can setup your global ~/.clojure/deps.edn (on linux and macOS) or %USERPROFILE%\.clojure\deps.edn (on windows) like this :
{...:aliases{:1.12-storm{:classpath-overrides{org.clojure/clojurenil}:extra-deps{com.github.flow-storm/clojure{:mvn/version"1.12.4"}com.github.flow-storm/flow-storm-dbg{:mvn/version"4.5.9"}}};; Optional plugins you find yourself using regularly:fs-web-plugin{:extra-deps{com.github.flow-storm/flow-storm-web-plugin{:mvn/version"1.0.0-beta"}}:jvm-opts["-Dclojure.storm.instrumentOnlyPrefixes.webPlugin=org.httpkit.server,ring.adapter.jetty,next.jdbc.result-set""-Dflowstorm.plugins.namespaces.webPlugin=flow-storm.plugins.web.all"]}...}}
Then you can start your repls with the :1.12-storm alias (like clj -A:1.12-storm). When the repl comes up evaluate the :dbg keyword to bring up the UI,
then click on Help→Tutorial on the menu for a tour of the basics.
You can setup your global ~/.lein/profiles.clj (on linux and macOS) or %USERPROFILE%\.lein\profiles.clj (on windows) like this :
{:1.12-storm{:dependencies[[com.github.flow-storm/clojure"1.12.4"][com.github.flow-storm/flow-storm-dbg"4.5.9"]]:exclusions[org.clojure/clojure]};; Optional plugins you find yourself using regularly:fs-web-plugin{:dependencies[[com.github.flow-storm/flow-storm-web-plugin"1.0.0-beta"]]:jvm-opts["-Dclojure.storm.instrumentOnlyPrefixes.webPlugin=org.httpkit.server,ring.adapter.jetty,next.jdbc.result-set""-Dflowstorm.plugins.namespaces.webPlugin=flow-storm.plugins.web.all"]}...}
Then you can start your project repls with +1.12-storm profile (like lein with-profile +1.12-storm repl). When the repl comes up evaluate the :dbg keyword to bring up the UI,
then click on Help→Tutorial on the menu for a tour of the basics.
Note
Running lein repl without a project
For some reason if you run lein with-profile +1.12-storm repl outside of a project it will not run with the profile
activated correctly.
If your project is using deps.edn, you can add an alias that looks like this :
{...:aliases{:1.12-storm{;; for disabling the official compiler:classpath-overrides{org.clojure/clojurenil}:extra-deps{com.github.flow-storm/clojure{:mvn/version"1.12.4"}com.github.flow-storm/flow-storm-dbg{:mvn/version"4.5.9"}}}}}
Once you have setup your deps.edn, start your repl with the :1.12-storm alias and run the debugger by evaluating
the :dbg keyworkd on your repl (this means just type :dbg and hit return).
If it is your first time using FlowStorm, when the UI comes up click on Help→Tutorial on the menu for a tour of the basics.
Once you have setup your lein profile globally or per project, start your repl with the 1.12-storm profile and run the debugger by evaluating
the :dbg keyworkd on your repl (this means just type :dbg and hit return).
Make sure you activate the profile with lein with-profile +1.12-storm repl.
If it is your first time using FlowStorm, when the UI comes up click on Help→Tutorial on the menu for a tour of the basics.
If you are using lein < 2.11.0 make sure your global :dependencies don’t include the official org.clojure/clojure dependency.
Moving to lein latest version should work ok even if your global :dependencies contains the Clojure dep.
If you use the clojure cli you can start a repl with the FlowStorm dependency loaded like this :
;; on Linux and OSX
clj -Sforce-Sdeps'{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}';; on Windows
clj -Sforce-Sdeps'{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version """4.5.9"""}}}'
If you are a lein user add the dependency to your project.clj :dependencies and run lein repl.
Then require the api namespace and start the debugger :
user>(require'[flow-storm.api:asfs-api]);; the only namespace you need to requireuser>(fs-api/local-connect);; will run the debugger GUI and get everything ready
You should now see a empty debugger window. Click on the recording button to leave the debugger in
recording mode and the let’s debug something:
user>#rtrace(reduce+(mapinc(range10)));; #rtrace will instrument and run some code
After running it, you should get the return value of the expression (as if #rtrace wasn’t there),
but now you will also have the debugger UI showing your recordings.
From here you probably want to check out the Flows tool which contains a lot of information
about exploring your recordings.
Debugging ClojureScript is a case of remote debugging in FlowStorm. This means the debugger
will run in a separate process and connect to the debuggee (your browser or nodejs runtime) via a websocket and optionally
an nrepl server.
There are two ways of using FlowStorm with ClojureScript :
With ClojureScriptStorm (recommended) : Swap your ClojureScript compiler by ClojureScriptStorm at dev and get everything instrumented automatically
Vanilla FlowStorm : Just add FlowStorm to your dev classpath and instrument by tagging and re-evaluating forms
ClojureScriptStorm is a fork of the official ClojureScript compiler that adds automatic instrumentation so you don’t need to think about it (you can still disable it when you don’t need it).
You use it by swapping the official ClojureScript compiler by ClojureScriptStorm at dev time, using dev aliases or profiles.
Note
Repl connection
For enabling every debugger feature, FlowStorm needs to connect to a cljs repl.
Currently only shadow-cljs repl over nrepl is supported.
For ClojureScript 1.11.* shadow-cljs >= 2.25.4, For ClojureScript 1.12.* shadow-cljs >= 3.1.1
FlowStorm >= 3.7.4
For setting up FlowStorm with shadow-cljs you need to modify two files, your shadow-cljs.edn and your deps.edn.
This is setup once and forget, so once you have configured FlowStorm you can do everything from the UI, without
any other sources modifications.
If you want a shadow-cljs template to play with, take a look at this repo.
Note
shadow-cljs
Currently you can only use ClojureScriptStorm with shadow-cljs if you are resolving your
dependencies with deps.edn. This means having :deps true or similar in your shadow-cljs.edn.
If you have your dependencies directly in your shadow-cljs.edn you will have to use Vanilla FlowStorm
for now.
This is because there is currently no way to swap the ClojureScript compiler in shadow-cljs.edn.
First, make your shadow-cljs.edn looks something like this :
So, the important parts are: you need to tell shadow to apply your deps.edn :1.12-cljs-storm alias, set up a nrepl port,
and also add flow-storm.storm-preload to your preloads. If you have other preloads make sure flow-storm.storm-preload
is the first one.
Then, modify your deps.edn dev profile to look like this :
{...:aliases;; this alias can be defined globally in your ~/.clojure/deps.edn so you don't have to modify this file in your project{:1.12-cljs-storm{:classpath-overrides{org.clojure/clojurescriptnil};; disable the official compiler:extra-deps{thheller/shadow-cljs{:mvn/version"3.3.4":exclusions[org.clojure/clojurescript]};; bring ClojureScriptStormcom.github.flow-storm/clojurescript{:mvn/version"1.12.134-3"};; add FlowStorm runtime depcom.github.flow-storm/flow-storm-inst{:mvn/version"4.5.9"}}}}}
There are lots of things going on there, but the main ones are: disabling the official compiler, adding
ClojureScriptStorm and FlowStorm dependencies, and then configuring what you want ClojureScriptStorm to automatically
instrument.
By default the JVM property cljs.storm.instrumentAutoPrefixes is true so all your project top level namespaces
will be instrumented automatically.
If you need to set that property to false it is important to configure what namespaces you want to instrument,
and you do this by setting the cljs.storm.instrumentOnlyPrefixes jvm property.
This is a comma separated list of namespaces prefixes, you normally want your app namespaces plus some libraries, like :
cljs.storm.instrumentOnlyPrefixes=org.my-app,org.my-lib,hiccup
And this is it. Once you have it configured, run your shadow watch as you normally do, load your app on the browser (or nodejs).
Whenever your need the debugger, on a terminal run the ui with your shadow-cljs.edn data :
Since we started the app with flowstorm.startRecording=false you will have to click on the record button once to start recording.
Whenever recording is enable and something executes under an instrumented namespace you should see the recordings appear in the debugger
under the main thread.
Note
recording expressions typed on the repl
If you type at the repl something like (defn foo [a b] (+ a b)) under an instrumented ns, the foo function will get instrumented
automatically and you will able to explore the recordings after the function is called.
On the other side, typing a simple expression like (+ 1 2) will not show anything, this is currently a limitation but you can
still make that work by wrapping the expression on a fn and immediately calling it, like fn [] (+ 1 2)
If you run the command above you are running cljs.main --repl which will start a ClojureScript repl on
your terminal and open a browser connected to it. You runtime will also start with FlowStorm preloaded and everything under cljs.user
is going to be instrumented.
Then on a different terminal run the FlowStorm UI :
And now refresh your browser page so your browser app connects to the UI.
Note
Limitations
There are some small limitations like not being able to modify instrumentation from the UI without restarting the repl.
This is because the FlowStorm UI needs to also connect via nrepl to JVM process running the compiler, which isn’t available
when running cljs.main.
Let’s say you are using shadow-cljs to start a ClojureScript repl.
First you need to add FlowStorm dependency to your project dependencies, like this :
$catshadow-cljs.edn{...:dependencies[...[com.github.flow-storm/flow-storm-inst"4.5.9"]];; the next two lines aren't needed but pretty convenient:nrepl{:port9000}:my-build-id{:devtools{:preloads[flow-storm.preload]}}...}
Then let’s say you start your repl like :
npx shadow-cljs watch :my-build-id
shadow-cljs - config: /home/jmonetta/demo/shadow-cljs.edn
shadow-cljs - server version: 2.19.0 running at http://localhost:9630
shadow-cljs - nREPL server started on port 9000
shadow-cljs - watching build :my-build-id
[:my-build-id] Configuring build.
[:my-build-id] Compiling ...
[:my-build-id] Build completed. (127 files, 0 compiled, 0 warnings, 6.19s)
cljs.user=>
As you can see from the output log shadow-cljs started a nrepl server on port 9000, this is the port FlowStorm needs to connect to,
so to start the debugger and connect to it you run :
;; on linux and mac-os
clj -Sforce-Sdeps'{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}'-X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-build-id
;; on windows
clj -Sforce-Sdeps'{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version """4.5.9"""}}}'-X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-build-id
And that is all you need, the debugger GUI will pop up and everything will be ready.
Try tracing some code from the repl :
cljs.user>#rtrace(reduce+(mapinc(range10)));; #rtrace will instrument and run some code
After running it, you should get the return value of the expression (as if #rtrace wasn’t there).
The debugger thread list (the one on the left) shows all the threads it has recordings for. Because we are
in javascript land there will always be just one thread, called main.
Double clicking it should open the "thread exploring tools" for that thread in a new tab.
This guide will cover all the tools in more detail but if you are interested in code stepping for example you will find
it in the code stepping tool at the bottom left corner of the thread tab, the one that has the () icon.
Click on it and use the stepping controls to step over the code.
Now that everything seems to be working move on and explore the many features FlowStorm provides. There are many ways of instrumenting
your code, and many ways to explore its executions.
If you are not using a repl or the repl you are using isn’t supported by FlowStorm yet you can still use the debugger
but not all features will be supported (mainly the browser features).
For this you can start the debugger like before but without any parameters, like this :
And then go to your app code and call (flow-storm.runtime.debuggers-api/remote-connect) maybe on your main, so every time your program starts
will automatically connect to the repl.
Note
ClojureScript environments
FlowStorm is supported for ClojureScript in :
Browsers
NodeJS
React native
Note
NodeJs and react-native
On NodeJs and react-native you need to install the websocket library.
Do this by running npm install websocket --save
For react-native if your app is running inside a cellphone you will have to also provide the :debugger-host key
to flow-storm.debugger.main/start-debugger with your box ip address, unless you are using adb reverse with your ports for
which you will have to adb reverse tcp:7722 tcp:7722 (the debugger websocket port)
Note
App initialization debugging
If you need to debug some app initialization, for adding #trace tags before the debugger is connected you
will have to require flow-storm.api yourself, probably in your main. All the tracing will be replayed to the debugger
once it is connected.
You can setup FlowStorm to debug multiple ClojureScript builds. This can be useful when your application is made up of multiple parts,
like when you have web workers.
Debugging multiple builds require multiple debugger instances, one per build.
The FlowStorm UI will start a websocket server, by default on 7722, so if you want to run multiple instances of it, you need
to run each instance under a different port. You can do this by providing a :ws-port to the startup command.
So let’s say you want to run two debuggers, one for your page and one for a webworker, your can run them like this :
# on one terminal start your app debugger instance
clj -Sforce-Sdeps'{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}'-X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-app :ws-port 7722
# on a second terminal start your webworker debugger instance
clj -Sforce-Sdeps'{:deps {com.github.flow-storm/flow-storm-dbg {:mvn/version "4.5.9"}}}'-X flow-storm.debugger.main/start-debugger :port 9000 :repl-type :shadow :build-id :my-web-worker :ws-port 7733
Now you also need to configure your builds to tell them what port they should connect to.
You do this by writing different preloads for each of your builds, and then using them instead of your flow-storm.storm-preload, like:
You can change the theme or customize the styles of different instances to make it easier to
know which debugger instance is connected to which application.
First we need to generate a deps.edn by running bb print-deps > deps.edn
Then modify the resulting deps.edn to add the FlowStorm alias like this :
{...:aliases{:dev{:classpath-overrides{org.clojure/clojurenil};; for disabling the official compiler:extra-deps{com.github.flow-storm/clojure{:mvn/version"1.12.4"}com.github.flow-storm/flow-storm-dbg{:mvn/version"4.5.9"}}:jvm-opts["-Dclojure.storm.instrumentOnlyPrefixes=user"]}}}
With clojure.storm.instrumentOnlyPrefixes=user we are telling ClojureStorm to instrument everything inside
the user namespace since the script doesn’t contain any namespace declaration.
And that is it, you can now start your clojure repl as usual, with clj -A:dev and then eval the :dbg keyword to
start the debugger UI.
Then eval the entire file to compile everything. To start the server in this example you will have to remove the wrapping
that is basically only allowing the server to run if we are running from babashka, like this :
The toolbar as well as the menu provides quick access to some general commands.
From left to right:
Cancel current running task. Whenever you a running a task that can take some time, this button will be red, and you can use it to cancel the task.
The Inst enable button allows to enable/disable instrumentation when in a Storm environment. A change on instrumentation will only affect newly compiled code.
The Flows vertical tab contains a bunch of tools for recording and analyzing your programs executions.
First of all, what are Flows?
A Flow is an "execution flow" recording unit. The only purpose of a flow is to group recording activity.
This grouping allows us for example to run some code and record it under flow-0, then modify our code, run it again, and
record this second run (or flow) under flow-1. Now we can access both recordings separately.
When you first open FlowStorm UI you will see four things, from left to right :
Clear your recordings if any.
Start/Stop recording. You can keep your heap from growing by stopping recording when you don’t need it.
Start/Stop recording the multi-thread timeline. Check out the multi-thread timeline tool.
The Rec on combo-box to select under what flow new recordings are going to be stored.
Whenever there is something recorded for a flow, a new tab with the flow name will appear.
Execution inside a flow will be grouped by threads. So the first thing you will see on a flow is a menu of threads
we have recordings for so far. This threads will be referred sometimes as timelines, since they are a sequence of
recorded execution steps.
Let’s say for example we have selected to record under flow-1 and run some multi threaded code.
We are going to see something like this :
There is a lot going on in the screenshot above, but the most important are :
we have configured FlowStorm to record new executions under flow-1
we have recorded stuff under flow-1 and there are also some previous recordings under flow-0
we are currently looking at flow-1, we have opened to explore the thread with id 1 called main and we are exploring it in the code stepper
Threads [4] indicates we have recorded activity in 4 threads, which we can access via this menu
Now for a different example :
This second image shows us exploring the recordings of a thread with id 474, called pool-4-thread-4 on flow-0.
The Flows tool also contains a toolbar that contains the Quick jump box.
Use it for quickly opening the first recording of a function in the code stepper.
Will autocomplete the first 25 matches.
In the screenshot above we see analyzing the recordings in the code stepper but there are many tools to explore the recorded timelines,
which we are going to describe next.
The code tool is the first of the Flows tab. It provides most of the functionality found in a traditional debugger.
You can use it to step over each expression, visualize values, locals and more.
The code tool allows you to step and "travel throught time" in two ways:
Use the controls at the top to step over your code in different ways.
Click on the highlighted forms to position the debugger at that point in time.
For moving around using the controls we have two rows of buttons.
The second row of controls, the most important one, are the stepping controls.
From left to right they are :
Step over backwards, will make one step backwards always staying on the same frame.
Step backwards, will step backwards in time going into sub functions.
Step out, will position the debugger in the next step after this function was called.
Step forward, will step forward in time going into sub functions.
Step over forward, will make one step forwards always staying on the same frame.
The numbers at the center show current_step_index / total_steps. This means that a total of total_steps has been recorded
for this thread so far. Write any number (less than total_steps) on the text box to jump into that position in time.
The buttons around the step counter are :
Jump to the first step of the recording.
Jump to the last step of the recording.
On the first row we have more controls, also for moving around in time.
The last stepping controls to the right are the power stepping controls.
Note
Highlighting
Only the forms that were executed at least once for the current function frame will be highlighted.
This means that code can be un-highlighted for two reasons:
there isn’t any recording for that part of the code
there is a recording but doesn’t belong to this function frame.
In the contrived example above we see we are stepping the foo function. All inside this function
body is highlighted but the bodies of the two anonymous functions for mapping and reducing. This
will only get highlighted once you step into their bodies.
In this case you are sure there are recordings for these functions bodies because the reduce is
non lazy, so if you keep stepping eventually you will get into their bodies, but there is a faster way.
For this you can right click on any un-highlighted expression that you think there could be a recording for and
select Jump forward here.
This will make FlowStorm scan from the current point of the timeline searching forward for a value
recorded at that coordinate (if any) and move the stepper to that point in time.
You also have Jump to first record here which will scan from the beginning of the timeline and Jump backwards here
which will search backwards from the current position.
The controls at the right are power stepping controls. They provide more powerfull ways of stepping through the code.
Clicking on the first, back, next or last buttons will navigate the timeline using the selected power stepping tool in the dropdown.
There are currently six power stepping tools :
identity, will step to the prev/next value which identity is the same as the current value.
'equality', will step to the prev/next value which is equals (clojure equality) to the current value.
same-coord will step to the prev/next value for the same coordinate. This means it will move to the next recording in
the timeline for this exact place in the code you are currently in. You can also see it as take me to all the situations
when the current expression executed doesn’t matter how we got to it.
custom, allows you to provide a predicate, which will be used to find the next step.
If you define it like (fn [v] (map? v)) will make the power stepper step over all map values.
custom-same-coord, the same as custom but fixed on the current coordinate like same-coord.
identity-other-thread, will step to a position which identity is the same as the current value in a different thread.
Here the prev and next arrows do the same thing, it will just jump to the first position that matches this value on a
different thread. This has some limitations. If there are more than two threads working with this identity there is no way
of choosing which thread to go. If you need more control, checkout the programmable debugging
section, specially the find-expr-entry function.
fn-call, allows you to provide a function to step to.
Note
Custom stepping
Custom power stepping is only supported in Clojure now.
Power stepping automatically skips all values equals to :flow-storm.power-step/skip. This can be useful when combined
with snapshot-value as a way of ignoring some of them, which provides a way of sampling
tight loops like in games.
You can use the search tool to search over all your flow recorded expressions and then make the stepper jump to them.
You can find the search tool under More tools → Search.
This type of search will walk over the selected threads expressions, converting their values to strings with pr-str up to the selected level and depth
and then checking if the resulting string contains your provided query string.
Searching by data window value allows you to select any of the current data windows and will search for the current selected data window value
over the selected threads expressions values using identity.
Whenever you click a highlighted form that has been executed multiple times inside the same function call (any kind of loop),
instead of immediately jumping into it, FlowStorm will popup a menu, like in the picture below :
This is the loops navigation menu. It allows you to quickly move around interesting iterations of the loop.
The menu will display slightly different options depending on you current position. The [FIRST] … and [LAST] …
entries will always show, which allows you to quickly jump to the first and last iteration of the loop.
If you are currently before the loop, clicking into any expression inside the loop will show the first 20
values for the clicked expression.
If instead you are currently in a expression after the loop, clicking back to an expression inside the loop,
will show the last 20 values for the clicked expression.
Now if you are currently stepping inside the loop, clicking any other expression inside it will show you 10 values
before and 10 values after of your current position.
Clicking on any of this entries will take you to that position in time.
If this is not enough, and you want to see all the values taken by some expression along the loop, you can always
use the printer tool.
FlowStorm will capture all functions that didn’t return because an exception unwind the stack, even
when that exception was captured further and it didn’t bubble up.
When an unwind situation is recorded a combobox will show up in the toolbar, containing the functions names
together with the exceptions types. If you hover the mouse over any of them, a tooltip will display the exception message.
Clicking on any of them will position the stepper at that point in time so you can explore what happened before.
You can configure FlowStorm to automatically jump to exceptions with the Config menu by checking Auto jump to exception
which is disabled by default.
The Locals pane will show the value of each binding for a symbol at binding time, which are the same thing
no matter where you are in the current block when working with immutable objects, but not when working with mutable ones.
If what was bound was muttable in any way, you will be seeing the value at binding time, and not at current time.
The stack panel will always show the current stacktrace. Be aware that the stacktrace
only include functions calls that had been recorded, so if you aren’t recording everything
there will be gaps.
Double clicking on any of the stack entries will make the debugger jump to that point in time.
The value panel in the code tool always display a pretty print of the current expression value.
You can configure the print-level and print-meta for the pretty printing by using the controls at the top.
The value panel showing the current expression in the code stepper is a little bit special since it also
contains a data window tab which allows you to quickly navigate the value or give it custom
visualizations.
Clicking on the Actions→Goto file:line menu allows you to search and jump to the first recording of a expression
with a file and line, given that one exists.
It will ask you for a file and line in the format of <class-path-file-path>:<line>.
If you have a file like src/org/my_app/core.clj and you are interested in expressions evaluating on like 42
you should search like org/my_app/core.clj:42.
The call stack tree tool is the second one of the Flows tab. It allows you to see the execution flow by expanding its call stack tree.
The call stack tree is useful for a high level overview of a complex execution and also as a tool for quickly moving through time.
You can jump to any point in time by double clicking on a node or by right clicking and on the context menu selecting Step code.
Use the button at the top left corner of the tree tool to show the current frame of the debugger in the tree.
There are also two value panels at the bottom that show the arguments and return value for the currently selected function call.
Note
Disabling the call stack tree tool
The call stack tree tool can be enable/disable on the fly if you are not using it and performance is an issue,
since keeping it updated can be expensive.
You can disable it from the Config menu or via the flowstorm.callTreeUpdate=false JVM prop.
The functions tool is the third one of the Flows tab.
It shows a list of all traced functions sort by how many times the have been called.
Normal functions will be colored black, multimethods magenta and types/records protocols/interfaces implementations in green.
Together with the call stack tree it provides a high level overview of a thread execution, and allows you to
jump through time much faster than single stepping.
You can search over the functions list by using the bar at the top.
Clicking on the calls counter of any function will display all function calls on the right sorted by time.
Each line will show the arguments vector for each call, and their return value.
Use the check boxes at the top to hide some of the arguments.
Double clicking on any row in the functions call list will jump to the stepper at that point in time.
You can also use the args and ret buttons to open the values on the inspector.
You can use this tool to record, display and navigate a total order of your recordings in a timeline.
This can be used, for example, to visualize how multiple threads expressions interleave, which is sometimes useful to debug race conditions.
You enable/disable the multi-thread timeline recording using its button on the toolbar. Recording on the multi-thread
timeline will make your program execution a little slower so it is recommended to have it paused unless you need it.
When you have something recorded on the multi-thread timeline you access the tool from the top right corner.
As an example, let’s say you record the execution this function :
By opening the tool a window like this should pop up :
As you can see the timeline tool displays a linear representation of your expressions. Times flows from top to bottom and
each thread gets assigned a different color. Every time a function is called or returns you will see it under the Function
column, and for each expression executed you will see a row with its Expression and Value.
Double clicking any row will make your code stepper (on the main window) jump to the code at that point in time.
Note
Big recordings timeline
Rendering the timeline needs some processing to render each sub-form and print each value so be aware it could be slow
if you try it on big recordings.
There is also a Only functions? checkbox at the top that will retrieve only function calls and can be used to visualize
the threads interleaving at a higher level.
FlowStorm has a lot of functionality to replace printing to the console as a debugging method since most of the time it is pretty
inefficient. Nonetheless, sometimes adding a bunch of print lines to specific places in your code is a very powerful way
of understanding execution.
For this cases FlowStorm has the Printer tool, which allows you to define, manage and visualize print points, without the need
of re running your code. It will work on your recordings as everything else.
You can add and re run print points over your recordings as many times as you need. To add a print point, just right click on any
recorded expression.
It will ask you for a couple optional fields.
The Message format is the "println text". A message to identify the print on the printer output. Here you can use any text, in which you can
optionally use %s for the printed value, same as you would use it with format.
The Expression field can be use to apply a transformer function over the value before printing it. Useful when you want to see a part of the value.
After you add them, you can access the Printers tool by navigating to More tools → Printers.
The threads selector allows you to select the thread the prints are going to run on.
Leaving it blank will run prints over all threads recordings (checkout the notes for caveats).
Clicking the refresh button will [re]run the printing again over the current recordings.
You can tweak your prints at any time, like changing the print-length, print-level, message, transform-fn or just temporarily disable any of them.
When you are ok re-setting you prints, just click refresh and they will print again.
Double clicking on any printed line will jump to the Flows code tab, with the debugger pointed to the expression that generated the print.
Important
Multi-thread prints order
If you select All threads, and have a multi-thread timeline recording, then the printer will use it and you can use prints to debug threads
interleaving for example, but if you run your printers with All threads selected without a multi-thread timeline recording they will print
sorted by thread and not in the order they happened.
You add code bookmarks by adding the (bookmark) statement to your code, which optionally accepts a label.
The first time a bookmark statement is executed it will make the FlowStorm UI jump to it. Since this behavior
is similar to a debugger statement in languages like Javascript, it is also aliased as (debugger) so you can
use whichever you prefer.
Note
ClojureScript support
This is currently only supported when using ClojureScriptStorm >= 1.11.132-9
UI bookmarks are useful when you find yourself jumping around, trying to understand a complex execution. They enable
you to mark execution positions so you can come back to them later.
You can bookmark the current position by pressing the bookmark button in the code tool, next to your stepping controls.
It will ask you the bookmark description.
The outputs tool can be used instead of your normal IDE/Editor panel to visualize your evaluations
results, your taps outputs and your out and err streams writes (like printlns).
The advantages being :
Custom visualizations
Quick nested values navigation
Quick taps values navigation
Datafy nav navigation
Access to all previously tapped values
Access to the last 10 evaluated values (instead of just *1 and *2)
Ability to search tapped values in Flows
The taps visualization system works out of the box while the evals result and printing capture currently
depends on you using nrepl and starting with the flow-storm middleware. Checkout the outputs setup
section for instructions.
Note
ClojureScript support
Only the taps viewer is currently supported on ClojureScript. The last evaluations
and the out and err streams capture aren’t supported yet.
For using all the features in the Outputs tool you need to be using nrepl and start your repl with
flow-storm.nrepl.middleware/wrap-flow-storm middleware.
If you use Cider for example you can add it to cider-jack-in-nrepl-middlewares via customizing the global
value or by using .dir-locals.el.
Everytime FlowStorm starts, it will add a tap, so whenever you tap> something
it will show on the taps list.
Click on any value to display it on the top data window.
If the tapped value has also been recorded as an expression in Flows, you can right click on it
and run Search value on Flows to move the debugger to that point in time.
Note
Search value on Flows
Be aware that if the code that taps your value is something like (tap> :a-key) you won’t be able to jump
to it using this, because :a-key isn’t a value recorded by FlowStorm, while if the tapping
code is like (tap> some-bind) or (tap> (+ 2 3)) or the tapping of any other expression
you should be able to jump to it.
So if you want to use this functionality as a "mark" so you can quickly jump to different parts of
the recordings from the Taps tool, you can do it like (tap> (str :my-mark))
A #tap tag will also be available, which will tap and return so you can use it like (+ 1 2 #tap (* 3 4))
Use the clear button to clear the list.
There is also #tap-stack-trace. It will tap the current stack trace.
You can always define a var for the current value being shown on any data window by clicking the def button.
Clicking on it will raise a popup asking for a symbol name. If you don’t provide a fully qualified symbol
it will define the var under user or cljs.user if you are in ClojureScript.
A quick way to use it is to provide a short name, let’s say foo, and then access it from your
code like user/foo.
Data Windows support datafy nav out of the box. The data window will always be showing the result of
clojure.datafy/datafy of a value. For maps or vectors where keys provide navigation it will automatically
add a blue arrow next to the value.
Clicking on the value will just dig the data, while clicking on the blue arrow will navigate as with
clojure.datafy/nav applied to that collection on that key.
The eql-query-pprint visualizer allows you to explore your data "entities" by looking at subsets of it
using queries similar to datomic pull queries like in the screenshots above.
Note
Disable by default
The EQL query pprint is disable by default. To enable it call (flow-storm.runtime.values/register-eql-query-pprint-extractor).
By entities it means maps which contains only keywords as their keys. Every other collection
is just traversed.
This are all valid queries :
[*]
[:name]
[:name :age :vehicles]
[:name :age {:vehicles [:type]}]
[:name :age {:vehicles [?]}]
[:name {:vehicles [*]}]
[:name :age {:vehicles [:type {:seats [?]}]}]
[:name :age {:vehicles [:type {:seats [:kind]}]}]
[:name {:houses [:rooms]}]
The * symbol means include all keys, while the ? symbol means just list the keys, which helps
exploring big nested maps with many keys.
The important part there are :id, :pred, and :on-create.
The :id will be the one displayed on the visualizers dropdown, and re-registering a visualizer
with the same id will replace the previous one.
:pred is a predicate on the data extracted from values, it should return true if this visualizer
can handle the value.
And :on-create will be a function that receives this value and renders a java fx node.
The val passed to on-create will also contain two special keywords :
:flow-storm.debugger.ui.data-windows.data-windows/dw-id The id of the data windows it’s being draw on
:flow-storm.debugger.ui.data-windows.data-windows/preferred-size (could be :small)
Optionally you can provide :on-update and :on-destroy.
:on-update will receive values from the runtime via fsa/data-window-val-update. It will also get a handle to
the original value (the one that created the DataWindow) and whatever map was returned by :on-create.
:on-destroy will be called everytime a visualizer gets removed, because you swapped your current visualizer
or because you went back with breadcrums. It can be useful in case you need to clear resources created by
:on-create.
:pred and :on-create will not receive the original value but the extracted aspects of it after
all registered extractors run.
You can check the data available to your visualizer for a value in a data window by calling :
(viz/data-window-current-val:chess-board-dw)
If the data already extracted from your value is not enough for your visualizer you can register
another extractor.
In this case we are going to register and extractor that will only run for vals which are sets and
contains at least one element which is a map with :kind, :player and :pos.
The extracted data will be the entire board.
All ids of extractors that applied for a value will be appended under ::fs-values/kinds of the value
as you will see next.
Now we can register a visualizer that will show only on values which contains a :chess-board kind.
(import'[javafx.scene.layoutGridPane])(import'[javafx.scene.controlLabel])(viz/register-visualizer{:id:chess-board;; only be available if the chess-board data extractor run on this value:pred(fn[val](contains?(::fs-values/kindsval):chess-board));; use the chess/board info to render a board with java fx:on-create(fn[{:keys[chess/board]}](let[kind->sprite{:king"♚":queen"♛":rook"♜":bishop"♝":knight"♞":pawn"♟"}pos->piece(->>board(mapv#(vector(:pos%)%))(into{}))]{:fx/node(let[gp(GridPane.)](doall(for[row(range8)col(range8)](let[cell-color(if(zero?(mod(+col(modrow2))2))"#f0d9b5""#b58863"){:keys[kindplayer]}(pos->piece[rowcol])cell-str(kind->spritekind"")player-color(whenplayer(nameplayer))](.addgp(doto(Label.cell-str)(.setStyle(format"-fx-background-color:%s; -fx-font-size:40; -fx-text-fill:%s; -fx-alignment: center;"cell-colorplayer-color))(.setPrefWidth50))(intcol)(introw)))))gp)}))})
Using custom visualizers with ClojureScript (or other remote environments) is a little bit more involved.
Registering aspect extractors is exaclty the same, since they run on the runtime (browswer, node, etc), but
custom visualizers should be registered on the debugger process. For this you need to create your visualizers in some
namespace, let’s say on /dev/visualizers.clj, add the dev folder to your classpath and then running the debugger UI
with something like :
You can make any visualizer the default by calling add-default-visualizer which takes a predicate on the val-data (the same received by :on-create) and
a visualizer key, like this :
FlowStorm is a tracing debugger, which means it can record what is happening without the need of stopping
your programs execution. This is all fine but doesn’t cover every possible situation. There are
cases where recording everything is impractical, and even if you can start/stop recording whenever you want,
being able to automatically stop your threads at certain points is useful.
For these cases, FlowStorm has the ability to set thread breakpoints, which means to define points (functions)
in the execution of your program where you want your threads to wait.
While the threads are waiting you can explore what happened so far.
As soon as a thread hits a break function, if recording is on, it will be blocked, and a "Threads blocked" menu will show up in the UI.
You can use this menu to unblock different threads.
Once you are done, you can pause recording using the pause button in the main toolbar and un-block every thread.
You can define thread breakpoints in two ways :
Using the browser (like in the image below), you can navigate to any function and click on the Break button. This will block the calling
thread every time the selected function gets called.
Or you can also install a break by calling (flow-storm.api/break-at 'my-proj.core/some-fn)
Note
Conditional threads breakpoints
The break-at fn accepts a second argument where you can provide a predicate that will be called with the same arguments
of the function you are breaking. It will only break when the predicate returns true. If you don’t
provide a predicate it will default to (constantly true)
You can remove breakpoints by :
Clicking on the browser instrumentation list delete buttons
Calling flow-storm.api/remove-break to remove a single breakpoint
Calling flow-storm.api/clear-breaks to remove all breakpoints
FlowStorm gives you full access to its internal indexes from the repl in Clojure and ClojureScript.
These allows you to explore your recordings using Clojure and write small programs to analyze
them if what’s provided by the GUI is not enough.
Most of what is documented here is also documented in the flow-storm.runtime.indexes.api namespace docstring, which
you can retrieve by evaluating (doc flow-storm.runtime.indexes.api). In fact, this is the only namespace you need to
require on your repl in order to work with your recordings.
Let’s say you have recorded some execution and now you want to work with the recordings from the repl.
So first we require the api ns as ia.
(require'[flow-storm.runtime.indexes.api:asia])
Now from the UI, you can get the thread-id of your recordings (the number next to the thread name)
which you will need for accessing them from the repl.
In most cases converting all entries into maps with ia/as-immutable is enough, but if you want a little bit more
performance you can access entries information without creating a immutable map first.
Timelines entries are of 4 different kinds: FnCallTrace, FnReturnTrace, FnUnwindTrace and ExprTrace.
You can access their data by using the following functions depending on the entry :
All kinds :
as-immutable
fn-call-idx
ExprTrace, FnReturnTrace and FnUnwindTrace :
get-coord-vec
ExprTrace, FnReturnTrace :
get-expr-val
FnUnwindTrace :
get-throwable
FnCallTrace :
get-fn-name
get-fn-ns
get-fn-args
get-fn-parent-idx
get-fn-ret-idx
get-fn-bindings
You can also access the timeline as a tree by calling :
You can retrieve forms by form id with get-form and then use get-sub-form-at-coord and a coordinate.
Here is a little example :
;; retrieve some expression entry into expruser>(defexpr(->timeline(get3)ia/as-immutable))user>expr{:type:expr,:coord[221],:result4,:fn-call-idx2,:idx3};; retrieve the fn-call entry for our expruser>(deffn-call(->timeline(get(:fn-call-idxexpr))ia/as-immutable))user>fn-call{:type:fn-call,:fn-ns"dev-tester":fn-name"other-function",:form-id1451539897,...};; grab it's formuser>(defform(->fn-call:form-idia/get-form:form/form))user>form(defother-function(fn[ab](+ab10)));; lets look at the sub-form from form at our expr coordinateuser>(ia/get-sub-form-at-coordform(:coordexpr))a
You can remotely debug any Clojure application that exposes a nrepl server.
In terms of dependencies, the debuggee side should be setup the same as a normal local setup, with the optional change that you can use
flow-storm-inst instead of flow-storm-dbg, being the former a slimmed down version of the later one that
doesn’t contain some libraries used only by the UI, but using the full flow-storm-dbg is also ok.
assuming your remote process at my-debuggee-box.com has started a nrepl server listening on port 9000 and
that the debugger websocket server is running on the default port.
After the tunnel is established, you can run you debugger UI like this :
If you need to connect the debugger to a remote process without a ssh tunnel or you need to configure the websocket server port you can do it like this :
Sometimes you are not debugging across a network but you want to run the FlowStorm UI on a different process.
A couple of aliases that can help for this :
{:aliases;; for your system process{:runtime-storm{:classpath-overrides{org.clojure/clojurenil}:extra-deps{com.github.flow-storm/clojure{:mvn/version"1.12.4"}com.github.flow-storm/flow-storm-inst{:mvn/version"4.5.9"}}};; for the FlowStorm GUI process:ui-storm{:extra-deps{com.github.flow-storm/flow-storm-dbg{:mvn/version"4.5.9"}}:exec-fnflow-storm.debugger.main/start-debugger:exec-args{:port7888}}}};; set your nrepl port here!
With those aliases you can start your application process by adding :runtime-storm and that is it.
To start the FlowStorm UI, go to another terminal and run clj -X:ui-storm.
When recording an application’s execution (specially when using ClojureStorm or ClojureScriptStorm) it could happen
that your process starts running out of heap. This section documents some tools FlowStorm provides to deal with this situations.
Most of the time, having the recording paused and just enabling it right before executing the action you are interested in is enough, but
when it isn’t, here are some other options.
A common situation is to see some high frequency functions adding a lot of noise to your recordings. For example a MouseMove event processing
will generate a lot of recordings while you use your app. There are a couple of ways to limit your functions calls by thread. You can identify
this kind of functions with the functions tool.
One tool you can use in this situations is the flowstorm.threadFnCallLimits JVM prop.
For example, you can add "-Dflowstorm.threadFnCallLimits=org.my-app/fn1:2,org.my-app/fn2:4" so every time the system starts, limits will be set for
org.my-app/fn1 and org.my-app/fn2. The number next to them is the limit. When a function reaches the limit FlowStorm will stop recording calls
to it and all the functions down its callstack.
You can also modify the limits from your repl, by calling flow-storm.runtime.indexes.api/[add-fn-call-limit|rm-fn-call-limit|get-fn-call-limits].
In ClojureScript you need to call them via your cljs repl api.
These limits are per thread, so when a thread recording is created it will start with the current defined counters, and each time a function gets called
the counter will decrement. When it reaches zero the function and all functions calls under it will stop being recorded.
When you clear your threads you are also clearing its limit counters, so next time you record something new counters will be initialized from your
global limits definitions.
If you are tracing some code that ends up in a infinite loop the debugger will choke on
too many traces, making everything slow and your only option is probably to restart it.
For preventing this, FlowStorm provides a couple of fuse/breakers, on threads trace count and on heap limits.
They are off by default but you can enable it from the Config menu.
Let’s say you added a thread trace limit of 1000. If you now run any code where a thread generates more than a 1000 traces FlowStorm will only
record those first 1000 traces and then discard the rest as if recording is off for that thread, while it will keep recording threads that haven’t
reached the limit.
Your code will continue execution as normal, which you can break using
your normal editor breaking commands if its an infinite loop, but now you have recordings to look at what is going on.
You can set a limit of 0 to disable it again.
You can set this limits at startup via the JVM options "-Dflowstorm.threadTraceLimit=1000" and
"-Dflowstorm.throwOnLimit=true".
FlowStorm will retain all values pointers when code executes so you can analyze them later. This works great with immutable values but
when your code uses mutable values like this :
to provide FlowStorm a way of creating a snapshot of the mutable value.
Note
ClojureStorm
If you are using ClojureStorm evaluate the previous defmethod in a ns that is not being
instrumented to avoid an infinite recursion.
Be aware that this is tricky in multithreading situations, as always with mutable values.
Note
Automatic derefing
FlowStorm will automatically deref Atoms, Refs, Agents, Vars and all pending-realized derefables on tracing
so no need to implement flow-storm.runtime.values/snapshot-value for them.
Note
Snapshoting and nested values
Snapshoting only applies to direct references to mutable values.
For example if you have an atom inside a nested immutable collection, it will not be snapshoted every time
that collection expression is being recorded, because the value being recorded is not a reference to an atom.
If this is important to you, you can still define snapshot-value for clojure.lang.PersistentArrayMap, etc, and
walk it down snapshoting everything mutable inside.
Note
snapshot-value and memory footprint
Although snapshot-value was created as a way to deal with mutable values it can be used to replace any value by another in the
recordings, which can be useful in other situations like reducing memory footprint when you don’t need the entire
value to be recorded.
If you are using ClojureStorm or ClojureScriptStorm it is important to learn how to control what gets instrumented and
how to uninstrument things. You can configure what gets instrumented automatically on startup via JVM properties but also
change this while your repl is running without the need to restart it.
FlowStorm by default will automatically figure out what to instrument from your project, which
you can always disable by setting the -Dclojure.storm.instrumentAutoPrefixes=false.
Note
How are auto prefixes calculated?
When the process starts it will scan all source folders on the classpath (everything not inside a jar containing clojure files) and build
a set of all top level namespace. All namespaces under those will be instrumented.
Currently it doesn’t detect single level namespaces, like when you have src/core.clj, if this is your case use
instrumentOnlyPrefixes.
If you prefer to be explicit about what gets instrumented you can use the JVM property "-Dclojure.storm.instrumentOnlyPrefixes=YOUR_INSTRUMENTATION_STRING"
where YOUR_INSTRUMENTATION_STRING should be a comma separated list of namespaces prefixes like :
my-project.,lib1.,lib2.core
which means automatically instrument my-project.* (which includes all sub namespaces), all lib1.* and only everything under lib2.core
All this can be changed after without restarting your repl from FlowStorm browser.
You can turn instrumentation on an off by using the button on the toolbar.
Remember that the change of this setting will only be effective on newly compiled code.
Apart from instrumentOnlyPrefixes which you probably already know, there is instrumentSkipPrefixes which also
accepts a comma separated list of namespaces prefixes to skip, and instrumentSkipRegex with accepts a regex for
namespaces to skip. All these together allows you to instrument you whole app but some undesired namespaces.
The next important thing is to be able to enable/disable instrumentation and add/remove prefixes without restarting the
repl.
You can use the Browser tool to check and change on the fly the prefixes you configured in the previous section.
Right clicking any namespace will give you options for what level of a namespace you want to instrument.
On the bottom pane (instrumentations) you will see your current instrumentation configuration (if any). Here we can see that everything under
ring.middleware.anti-forgery will be instrumented every time something inside it gets compiled.
You can remove entries using the del buttons or temporarily disable/enable them using the Enable all checkbox.
You can use the Add menu in the picure above to add instrumentation prefixes.
After changing any prefix FlowStorm will ask if you want it to reload the affected namespaces for you. Namespace reloading will
all reload all namespaces it depends on in topological order, so it shouldn’t break your system in any way.
You can also provide functions to be called before and after reloading in case you need to stop and start your system with :
Just changing the prefixes without reloading will not make your currently loaded code [un]instrumented.
If you haven’t let FlowStorm reload them for you, you can always recompile them as usual with your editor commands or
by executing something like (require 'the-selected.namespace :reload).
Instructions here only apply to vanilla FlowStorm. If you are using ClojureStorm or ClojureScriptStorm (recommended)
this is done automatically for you, so just skip this section.
Code instrumentation in FlowStorm is done by rewriting your code, in a way that doesn’t change its behavior
but when executed will trace everything the code is doing.
You can instrument any top level form at the repl by writing #trace before it, like this :
#trace(defnsum[ab](+ab))
and then evaluating the form.
important
#trace is meant to be used with forms that don’t run immediately, like: defn, defmethod, extend-type, etc.
Use #rtrace to trace and run a form, like #rtrace (map inc (range 10)).
The first argument is a set of namespaces prefixes to instrument. In the previous example it means
instrument all namespaces starting with org.my-app.core, and all starting with cljs.
The second argument can be a map supporting the following options :
:excluding-ns a set of strings with namespaces that should be excluded
:disable a set containing any of #{:expr:binding:anonymous-fn} useful for disabling unnecessary traces in code that generate too many
These are some limitations when instrumenting forms :
Very big forms can’t be fully instrumented. The JVM spec has a limit on the size of methods and instrumentation adds a lot of code.
When instrumenting entire namespaces, if you hit this limit on a form a warning will printed on the console saying Instrumented expression is too large for the Clojure compiler
and FlowStorm automatically tries to instrument it with a lighter profile, by disabling some instrumentation.
Functions that call recur without a loop
Functions that return recursive lazy sequences. Like (fn foo [] (lazy-seq (… (foo))))
Most of the time you can instrument code by just clicking in the browser. The only exceptions are functions that were just defined in the repl
and weren’t loaded from a file.
Using the browser you can also instrument multiple namespaces. Do this by selecting the namespaces you are interested in
and then a right click should show you a menu with two instrumentation commands.
Instrument namespace :light - record function arguments and return values (not expressions, no bindings tracing)
Light instrumentation is useful when you know the functions generate too many traces, so you can opt to trace just functions
calls and returns. You can then fully instrument whatever functions you are interested in.
If you have instrumented a form with the :light profile you can fully instrument it by right clicking on the current form
and then clicking Fully instrument this form.
For using a pluggin follow each plugging instructions which should normally consists of adding
its dependency and then setting the jvm prop flowstorm.plugins.namespaces with all the main
namespaces of the plugins you want loaded at startup, like : "-Dflowstorm.plugins.namespaces=flow-storm.plugins.my-plugin.all"
After that you should see a new vertical tab with the plugin UI as you can see here :
The runtime code that will analyze the recordings and expose an api for the UI
The UI component which will visualize and interact with the data via the runtime api
This split is not required, but it is important if you want your plugin to support ClojureScript also or
remote Clojure debugging where the UI is not running in the same process as the runtime.
This components are normally split in two files, a runtime.clj and ui.clj, but you can name them however you
want.
We are going to go over each part in more detail but for a real plugin please checkout the core.async.flow plugin.
(nsflow-storm.plugins.my-plugin.runtime(:require[flow-storm.runtime.indexes.api:asia][flow-storm.runtime.debuggers-api:asdbg-api]))(defnmy-data-extraction[flow-idthread-id](let[timeline(ia/get-timelineflow-idthread-id)](reduce(fn[acctl-entry];; extract some interesting info from the timeline){}timeline)));; Expose your function so it can be called from the UI part(dbg-api/register-api-function:plugins.my-plugin/extract-datamy-data-extraction)
(nsflow-storm.plugins.my-plugin.ui(:require[flow-storm.debugger.ui.plugins:asfs-plugins][flow-storm.debugger.runtime-api:asruntime-api:refer[rt-api]])(:import[javafx.scene.controlLabel]))(fs-plugins/register-plugin:my-plugin{:label"My plugin":css-resource"flow-storm-my-plugin/dark.css":dark-css-resource"flow-storm-my-plugin/dark.css":light-css-resource"flow-storm-my-plugin/light.css":on-focus(fn[{:keys[some-other-data]}];; This gets called everytime the plugin tab gets focused):on-create(fn[_]{:fx/node(Label.;; You can call your runtime registered function(str(runtime-api/call-by-fn-keyrt-api:plugins.my-plugin/extract-data[010]))):some-other-data42}):on-flow-clear(fn[flow-id{:keys[some-other-data]}];; this gets called everytime a flow is discarded so you can update your plugin UI accordignly)})
As you saw in the ui plugin registration, you can provide three resources related to styling :
:css-resource If there is any, it will be loaded and applied. Here is where you put your JavaFX pluging styles
:dark-css-resource This styles are going to be applied only in dark mode
:light-css-resource This styles are going to be applied only in light mode
For making sure you plugin styles doesn’t mix with other styles, your plugin is automatically wrapped in a pane
with your plugin key (my-plugin in the example above) as a class.
This means your plugin css can contain code like :
All functions that start the debugger ui (flow-storm.api/local-connect, flow-storm.debugger.main/start-debugger) accept a map
with the :styles, :title and :theme keywords. If :styles points to a css file it will be used to overwrite the default styles, in case you
want to change colors, make your fonts bigger, etc. :theme could be one of :auto (default), :light, :dark. Title can be used to
distinguish between multiple debugger instances.
Debugging ClojureScript react native application needs a combination of ClojureScript and remote debugging.
Assuming you are using shadow-cljs, have added the flow-storm-inst dependency, and that it started a nrepl server on port 9000, you
can start a debugger and connect to it by running :
For those using current versions of WSL2 on Windows it should be pretty straight forward.
export DISPLAY=:0
export WSL2_GUI_APPS_ENABLED=1
Font issues had been reported on some distros, like java.lang.NullPointerException: Cannot read field "firstFont" because "<local4>" is null which
seams to be solved just by installing font packages like dejavu-fonts or ttf-dejavu depending on the distro.
You can add this two jvm options to tell FlowStorm how to open forms in files and inside jars :
flowstorm.jarEditorCommand : a command with optional <<JAR>>, <<FILE>> and <<LINE>> placeholders
flowstorm.fileEditorCommand : a command with optional <<FILE>> and <<LINE>> placeholders
If you define those, clicking on your forms namespaces link in the code tool should run the provided commands.
On expressions sub-forms that contains line meta you should also be able to right click and select "Open in editor"
which should open the file at that specific line (useful for long forms).
Here are some known setups for most common editors :
;; for opening your project files"-Dflowstorm.fileEditorCommand=emacsclient -n +<<LINE>>:0 <<FILE>>";; simple way for opening files inside jars (works on linux only)"-Dflowstorm.jarEditorCommand=emacsclient -n +<<LINE>>:0 <<JAR>>/<<FILE>>";; for opening files inside jars that works on every OS (requires FlowStorm >= 3.17.3)"-Dflowstorm.jarEditorCommand=emacsclient --eval '(let ((b (cider-find-file \"jar:file:<<JAR>>!/<<FILE>>\"))) (with-current-buffer b (switch-to-buffer b) (goto-char (point-min)) (forward-line (1- <<LINE>>))))'"
FlowStorm works fine with most macros, except the ones that don’t preserve meta at macroexpansion, which FlowStorm needs in their absence,
can cause GAPS IN EXECUTION TRACING AND PREVENT IT FROM LINKING THE EXECUTED CODE BACK TO THE ORIGINAL SOURCE.
When macros are involved, the forms compiled by the Clojure compiler aren’t the ones on your source files, but the ones generated
by macro-expanding all the macros. In order to link the compiled forms back to the forms in your source code, for each instrumented form,
right after the form is read by the reader, FlowStorm will walk the form down, annotating with meta each sub-form with a coordinate,
which will then be used after macro expansion to link a compiled expression back to your source code.
Macros can be as simple as code reorganizing ones (like →, when, and, defn, etc) or whole compilers like clojure.core.async/go and
Electric, so it really depends on the macros.
If you see code inside a macro not being traced feel free to report an issue, there is nothing FlowStorm can do from its side but
we can work together with the macro developer making sure it preserves all meta after macro expansion, which sometimes may be possible.
The Locals pane will show the value of each binding for a symbol AT BINDING TIME, which is the same thing
no matter where you are in the current block when working with immutable objects, BUT NOT WHEN WORKING WITH MUTABLE ONES.
If what was bound was mutable in any way, you will be seeing the value at binding time, and not at current time which
could cause some confusion.
The locals pane will only display bindings for the current function. Locals visible from the current function but not defined
in it (like in the case of closures) aren’t shown.
Currently we can’t instrument clojure.core or FlowStorm itself since they endup in infinite tracing recursions.
This can be solved, but it is currently a limitation.
because l is a primitive long, the compiler can replace the (zero? l) with intrinsics (LCONST_0, LCMP, IFNE) so the
(zero? l) isn’t a expression anymore, just a statement. In these cases you will see the if test return un-highlighted, but
you can still tell which branch the code went thru because the chosen branch will be the highlighted one.
This is because most editors, (specially via nrepl) eval some other code on your namespace in order to load the
contents of your file. If that namespace is instrumented this will be also recorded, even when it is probably not
of your interest.
This is harmless, just clear your recordings before running and recording anything else.
If you follow the best practice of start recording right before running the stuff you are interested in recording you
should never see this.
When you are evaluating some code that macroexpands to a (do form-1 … form-n) the compiler recursively calls eval on the
sub forms. Because it is tricky in the compiler to tell apart your original source form from the ones the macroexpansion returned
those form-1 to form-n get instrumented and then traced as if they were on your code.
The tricky part is related to tooling like IDEs sometimes wrapping your forms in some macros that expand to a (do form-1 … form-n)
so we can’t simply stop instrumenting after that situation.
Checkout that you don’t have piggieback on the classpath dragged by some dependency. Currently
if piggieback is pressent FlowStorm will assume a ClojureScript repl in which the outputs panel
isn’t supported yet.
================================================
FILE: examples/plugins/basic-plugin/deps.edn
================================================
{}
================================================
FILE: examples/plugins/basic-plugin/src/flow_storm/plugins/timelines_counters/all.clj
================================================
(ns flow-storm.plugins.timelines-counters.all
(:require [flow-storm.plugins.timelines-counters.ui]
[flow-storm.plugins.timelines-counters.runtime]))
================================================
FILE: examples/plugins/basic-plugin/src/flow_storm/plugins/timelines_counters/runtime.cljc
================================================
(ns flow-storm.plugins.timelines-counters.runtime
(:require [flow-storm.runtime.indexes.api :as ia]
[flow-storm.runtime.debuggers-api :as dbg-api]))
(defn timelines-counts [flow-id]
(reduce (fn [acc [fid tid]]
(if (= flow-id fid)
(let [timeline (ia/get-timeline flow-id tid)]
(assoc acc tid (count timeline)))
acc))
{}
(ia/all-threads)))
(dbg-api/register-api-function :plugins.timelines-counters/timelines-counts timelines-counts)
================================================
FILE: examples/plugins/basic-plugin/src/flow_storm/plugins/timelines_counters/ui.clj
================================================
(ns flow-storm.plugins.timelines-counters.ui
(:require [flow-storm.debugger.ui.plugins :as fs-plugins]
[flow-storm.debugger.runtime-api :as runtime-api :refer [rt-api]]
[clojure.string :as str])
(:import [javafx.scene.control Label Button TextField]
[javafx.scene.layout HBox VBox]
[javafx.event EventHandler]
[javafx.scene Node]))
(fs-plugins/register-plugin
:timelines-counter
{:label "Timelines counter"
:on-create (fn [_]
(let [counts-lbl (Label. "")
flow-id-txt (TextField. "0")
refresh-btn (doto (Button. "Refresh")
(.setOnAction
(reify javafx.event.EventHandler
(handle [_ _]
(->> (runtime-api/call-by-fn-key rt-api
:plugins.timelines-counters/timelines-counts
[(parse-long (.getText flow-id-txt))])
(mapv (fn [[thread-id cnt]]
(format "ThreadId: %d, Timeline Count: %d" thread-id cnt)))
(str/join "\n")
(.setText counts-lbl))))))
tools-box (HBox. (into-array Node [(Label. "FlowId:") flow-id-txt refresh-btn]))
main-box (VBox. (into-array Node [tools-box
counts-lbl]))]
{:fx/node main-box}))})
================================================
FILE: llm-prompt.txt
================================================
You can use the FlowStorm debugger tools via the repl to analyze FlowStorm recordings which records Clojure systems execution, for
debugging or gathering extra information that can only be known at runtime.
Recordings are grouped into flows. A flow just groups related recordings of some system's execution.
Each flow contains a group of timelines, which are arrays containig the recorded executions steps for a thread.
On each recorded flow there is one timeline per thread. If the recorded program executed in multiple threads,
each of the timelines can be accesed by flow-id and thread-id.
Each timeline entry is a Clojure map of one of the following :type :
- :fn-call A entry representing a function call
- :fn-return A entry representing a function return
- :fn-unwind A entry representing a function that throwed and exception instead of returning
- :expr-exec A entry representing a expression execution.
Next I'll list all the functions you have available for retrieving and exploring the timlines. They allow you to
write small Clojure programs to inspect the timelines for debugging or understanding anything about a Clojure program's execution.
You can get all the recorded flows with their threads ids by calling :
(flow-storm.runtime.indexes.api/all-flows)
which will return a Clojure map from flow-id to a vector of threads ids.
For each `thread-id` you can retrieve its timeline calling :
(flow-storm.runtime.indexes.api/get-referenced-maps-timeline flow-id thread-id)
You can bound each timeline to a name with `def` once and then work with them.
Since timelines implement most Clojure collections interfaces, you can call get, count, map, take, filter, reduce, etc on them.
For example, assuming you have bound a timeline to `tl`, you can get the entry at position 5 on the timeline by evaluating :
(get tl 5)
If the entry is of `:fn-call` type it will contain the following keys :
- :type Will be :fn-call
- :fn-name A string containing the name of the function being called
- :fn-ns A string containing the namespace of the function being called
- :form-id The id to the Clojure function form
- :fn-args-ref The value-id of the function arguments vector
- :parent-idx The timeline index of the parent function, or nil if this is the first funciton being called
- :ret-idx The timeline index of the fn-return step for this function call
If the entry is of `:expr` type it will contain the following keys :
- :type Will be :expr
- :result-ref The value-id of this expression value
- :fn-call-idx The index of the fn-call wrapping this step
- :form-id The id to the Clojure function form
- :coord The coordinate of this expression in the form represented by form-id
If the entry is of `:fn-return` type it will contain the following keys :
- :type Will be :fn-return
- :result-ref The value-id of this expression value
- :fn-call-idx The index of the fn-call wrapping this step
- :form-id The id to the Clojure function form
- :coord The coordinate of this expression in the form represented by form-id
If the entry is of `:fn-unwind` type it will contain the following keys :
- :type Will be :fn-unwind
- :throwable-ref The value-id of the Throwable/Exception object
- :form-id The id to the Clojure function form
- :coord The coordinate of this expression in the form represented by form-id
Using the :fn-call (:parent-idx, :ret-idx), and the :fn-call-idx of the other entries you can also
navigate the timeline as a graph.
You can call `flow-storm.runtime.indexes.api/get-form-at-coord` with a form-id and a coordinate to retrieve the Clojure form for
a specific :form-id and :coord. You can use a nil coord to retrieve the outer form.
You can call `flow-storm.runtime.values/deref-val-id` with a value-id to get the Clojure value represented by that value id.
So for example given a value-id of 1, you can preview it with something like :
(binding [*print-level* 5
*print-length* 5]
(flow-storm.plugins.mcp.runtime/deref-val-id 1))
but you coan inspect the value however you need.
================================================
FILE: package.json
================================================
{
"dependencies": {
"websocket": "1.0.34",
"shadow-cljs": "2.23.3"
}
}
================================================
FILE: resources/flowstorm/fonts/LICENSE.txt
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: resources/flowstorm/styles/font-size-lg.css
================================================
.root {-fx-font-size: 19;}
.ikonli-font-icon {-fx-font-size: 21;}
.btn-sm {-fx-font-size: 16;}
.btn-xs {-fx-font-size: 13;}
.threads-tab-pane > .tab-header-area {-fx-padding: 5 0 0 158;}
.flows-tab-pane > .tab-header-area {-fx-padding: 10 0 0 370;}
.flows-combo {-fx-pref-width: 184; -fx-font-size: 18;}
================================================
FILE: resources/flowstorm/styles/font-size-md.css
================================================
.root {-fx-font-size: 16;}
.ikonli-font-icon {-fx-font-size: 17;}
.btn-sm {-fx-font-size: 13;}
.btn-xs {-fx-font-size: 11;}
.threads-tab-pane > .tab-header-area {-fx-padding: 5 0 0 134;}
.flows-tab-pane > .tab-header-area {-fx-padding: 10 0 0 315 ;}
.flows-combo {-fx-pref-width: 155; -fx-font-size: 15;}
================================================
FILE: resources/flowstorm/styles/font-size-sm.css
================================================
.root {-fx-font-size: 13;}
.ikonli-font-icon {-fx-font-size: 15;}
.btn-sm {-fx-font-size: 10;}
.btn-xs {-fx-font-size: 8;}
.threads-tab-pane > .tab-header-area {-fx-padding: 5 0 0 112 ;}
.flows-tab-pane > .tab-header-area {-fx-padding: 10 0 0 265 ;}
.flows-combo {-fx-pref-width: 127; -fx-font-size: 13;}
================================================
FILE: resources/flowstorm/styles/font-size-xl.css
================================================
.root {-fx-font-size: 22;}
.ikonli-font-icon {-fx-font-size: 23;}
.btn-sm {-fx-font-size: 19;}
.btn-xs {-fx-font-size: 16;}
.threads-tab-pane > .tab-header-area {-fx-padding: 5 0 0 180;}
.flows-tab-pane > .tab-header-area {-fx-padding: 10 0 0 420;}
.flows-combo {-fx-pref-width: 211; -fx-font-size: 21;}
================================================
FILE: resources/flowstorm/styles/styles.css
================================================
@font-face {
src: url('../fonts/Roboto-Light.ttf');
}
@font-face {
src: url('../fonts/Roboto-Medium.ttf');
}
.main-pane {
-fx-font-family: 'Roboto Medium';
}
.context-menu {
-fx-background-color: -fx-theme-base4;
}
.separator-label {
-fx-text-fill: -fx-theme-separator;
}
.tab .ikonli-font-icon {
-fx-icon-color: -fx-theme-button2-text;
-fx-background-color: linear-gradient(to top, -fx-base, derive(-fx-base,30%));
}
.button {
-fx-background-color: -fx-theme-button1-background;
-fx-text-fill: -fx-theme-button1-text;
}
.toggle-button:selected {
-fx-background-color: -fx-theme-on;
-fx-text-fill: #323232;
}
.list-view .list-cell:selected {
-fx-background-color: -fx-theme-list-selection;
-fx-text-fill: -fx-theme-flow-code-text;
}
.table-view .table-row-cell:selected {
-fx-background-color: -fx-theme-list-selection;
-fx-text-fill: -fx-theme-flow-code-text;
}
.combo-box .label {
-fx-text-fill: -fx-theme-button2-text;
}
.tab {
-fx-opacity: 0.5;
-fx-text-fill: -fx-theme-button1-text;
}
.hl-combo {
-fx-background-color: -fx-theme-hl-combos;
}
.important-combo {
-fx-background-color: -fx-theme-important-combos;
}
.tab: selected {
-fx-opacity: 1;
}
.vertical-tab:selected {
-fx-background-color: -fx-theme-selected-main-tools-tab-background;
}
.vertical-tab:selected .tab-label {
-fx-text-fill: -fx-theme-selected-main-tools-tab-text;
}
.button.thread-continue-btn {
-fx-background-color: -fx-theme-breakpoint-continue;
}
.thread-blocked {
-fx-text-fill: -fx-theme-warning;
}
.button.clear-break-btn {
-fx-background-color: -fx-theme-breakpoint;
}
.link-lbl {
-fx-text-fill: -fx-theme-links;
-fx-cursor: hand;
}
.link-lbl-no-color {
-fx-underline: true;
-fx-cursor: hand;
}
.light {
-fx-opacity: 0.3;
}
.button:hover {
-fx-background-color: -fx-theme-button-hover;
-fx-color: -fx-hover-base;
}
.button:focused { /* this is so buttons doesn't look smaller when they are focused */
-fx-background-insets: 0 0 -1 0;
}
.button .ikonli-font-icon {
-fx-icon-color: -fx-theme-button1-text;
}
.thread-refresh {-fx-background-color: -fx-theme-base1;}
.thread-refresh .ikonli-font-icon {-fx-icon-color: red;}
.mirrored {
-fx-scale-x: -1;
}
.text-field {
-fx-prompt-text-fill: -fx-theme-button2-text;
}
.ok {
-fx-background-color: -fx-theme-ok;
-fx-text-fill: #323232;
}
.warning {
-fx-background-color: -fx-theme-warning;
-fx-text-fill: #323232;
}
.fail {
-fx-background-color: -fx-theme-attention;
-fx-text-fill: #323232;
}
.attention {
-fx-background-color: -fx-theme-attention;
-fx-text-fill: #323232;
}
.tree-search {
-fx-background-insets: 1 0 1 0;
}
.form-pane {
-fx-background-color: -fx-theme-base2;
}
.dialog-pane {
-fx-background-color: -fx-theme-base2;
}
.form-pane.form-background-highlighted {
-fx-background-color: -fx-theme-form-highlight;
}
.label.defmethod {
-fx-text-fill: -fx-theme-defmethod-text;
}
.label.defn {
-fx-text-fill: -fx-theme-defn-text;
}
.label.extend-type {
-fx-text-fill: -fx-theme-extend-text;
}
.label.extend-protocol {
-fx-text-fill: -fx-theme-extend-text;
}
.label.anonymous {
-fx-text-fill: -fx-theme-defn-text;
}
.label.fn-ns {
-fx-text-fill: -fx-theme-dim-text;
}
.code-token {
-fx-font-family: 'monospaced';
-fx-fill: -fx-theme-flow-code-text;
}
.monospaced {
-fx-font-family: 'monospaced';
}
.code-token.executing {
-rtfx-background-color: -fx-theme-flow-code-executing-background;
-fx-fill: -fx-theme-flow-code-executing-text;
}
.code-token.executing-dim {
-rtfx-background-color: -fx-theme-flow-code-executing-background-dim;
}
.code-token.interesting {
-fx-fill: -fx-theme-flow-code-interesting-text;
-fx-cursor: hand;
-fx-font-weight: bold;
}
.code-token.possible {
-fx-cursor: default;
}
.forms-scroll-container {
-fx-padding: 10 0 0 0;
}
.form-pane {
-fx-padding: 10;
}
.thread-controls-pane {
-fx-background-color: -fx-theme-base2;
-fx-padding: 10;
}
.trace-position-box {
-fx-alignment: center;
}
.threads-tab-pane > .tab-header-area {
-fx-padding: 5 0 0 92 ;
}
.flows-tab-pane > .tab-header-area {
-fx-padding: 5 0 0 135 ;
}
.fn-call-list-cell {
-fx-background-color: -fx-theme-base2;
}
.fn-call-list-cell .label{
-fx-text-fill: -fx-theme-flow-code-text;
}
/*****************/
/* Browser stuff */
/*****************/
.label.browser-fn-fq-name {
-fx-font-weight: bold;
-fx-text-fill: -fx-theme-button1-background;
-fx-padding: 10;
}
.browser-fn-args-box {
-fx-padding: 10;
}
.label.browser-fn-attr {
-fx-padding: 10;
}
.button.browser-instrument-btn {
-fx-display: none;
visibility: hidden;
}
.button.browser-break-btn {
visibility: hidden;
-fx-background-color: -fx-theme-breakpoint;
}
.button.browser-break-btn:hover {
visibility: hidden;
-fx-background-color: -fx-theme-button-hover;
-fx-color: -fx-hover-base;
}
.button.browser-break-btn.enable {
visibility: visible;
}
.button.browser-instrument-btn.enable {
visibility: visible;
}
.browser-var-buttons {
-fx-padding: 10;
}
.browser-instr-type-lbl {
-fx-text-fill: -fx-theme-dim-text;
}
.browser-instr-tools-box {
-fx-padding: 10;
}
.main-bottom-bar-box {
-fx-background-color: -fx-theme-base1;
}
.button.reload-tree-btn {
-fx-background-color: -fx-theme-button2-background;
}
/****************/
/* Docs browser */
/****************/
.docs-fn-name {
-fx-font-weight: bold;
-fx-text-fill: -fx-theme-title-label;
-fx-padding: 5;
}
.docs-label {
-fx-text-fill: -fx-theme-field-label;
-fx-padding: 5;
}
.docs-example-box {
-fx-padding: 10;
}
.docs-box {
-fx-padding: 10;
-fx-background-color: -fx-theme-base2;
}
.docs-type-name {
-fx-text-fill: -fx-theme-extend-text;
}
.docs-arg-symbol {
-fx-text-fill: -fx-theme-dim-text;
}
.hidden-node {
-fx-display: none;
visibility: hidden;
-fx-max-width: 0;
}
.docs-example-ret-symbol {
-fx-text-fill: -fx-theme-links;
}
/*****************/
/* Timeline tool */
/*****************/
.timeline-tool .controls-box {
-fx-padding : 10;
}
/****************/
/* Data Windows */
/****************/
.data-window .breadcrums .button {
-fx-background-color: -fx-theme-breadcrums;
}
/***********/
/* Outputs */
/***********/
.outputs-dw {
-fx-background-color: -fx-theme-base2;
}
================================================
FILE: resources/flowstorm/styles/theme_dark.css
================================================
* {
-fx-theme-base1: #323232;
-fx-theme-base2: #3f474f;
-fx-theme-base3: #2f353b;
-fx-theme-base4: #717175;
-fx-theme-separator: orange;
-fx-theme-dim-text: #919191;
-fx-theme-button1-background: #336699;
-fx-theme-button1-text: white;
-fx-theme-button-hover: #9c0084;
-fx-theme-button2-background: #919191;
-fx-theme-button2-text: white;
-fx-theme-selected-main-tools-tab-background: #4743ba;
-fx-theme-selected-main-tools-tab-text: white;
-fx-theme-flow-code-text: #FFF;
-fx-theme-flow-code-executing-text: #3f474f;
-fx-theme-flow-code-executing-background: #00d97e;
-fx-theme-flow-code-executing-background-dim: #009c5a;
-fx-theme-flow-code-interesting-text: #eb34d5;
-fx-theme-defmethod-text: #de00c0;
-fx-theme-defn-text: #FFF;
-fx-theme-extend-text: #65ce8a;
-fx-theme-links: #de00c0;
-fx-theme-breadcrums: #de00c0;
-fx-theme-ok: #00ff00;
-fx-theme-on: #00d97e;
-fx-theme-attention: #ff0000;
-fx-theme-warning: orange;
-fx-theme-title-label: orange;
-fx-theme-field-label: #de00c0;
-fx-theme-breakpoint: #ff0000;
-fx-theme-breakpoint-continue: #60d61b;
-fx-theme-form-highlight: #636363;
-fx-theme-list-selection: #788da1;
-fx-theme-hl-combos: #555478;
-fx-theme-important-combos: #792a6c;
}
.root {
-fx-base: -fx-theme-base1;
-fx-background: -fx-theme-base1;
-fx-control-inner-background: -fx-theme-base1;
}
================================================
FILE: resources/flowstorm/styles/theme_light.css
================================================
* {
-fx-theme-base1: #ddd;
-fx-theme-base2: #eee;
-fx-theme-base3: #ebecff;
-fx-theme-base4: #ebecff;
-fx-theme-separator: #919191;
-fx-theme-dim-text: #919191;
-fx-theme-button1-background: #336699;
-fx-theme-button1-text: white;
-fx-theme-button-hover: #9c0084;
-fx-theme-button2-background: #919191;
-fx-theme-button2-text: #0b0d2e;
-fx-theme-selected-main-tools-tab-background: #4743ba;
-fx-theme-selected-main-tools-tab-text: white;
-fx-theme-flow-code-text: #3f474f;
-fx-theme-flow-code-executing-text: #3f474f;
-fx-theme-flow-code-executing-background: #43e8a4;
-fx-theme-flow-code-executing-background-dim: #9de3c6;
-fx-theme-flow-code-interesting-text: #eb34d5;
-fx-theme-defmethod-text: #9c0084;
-fx-theme-defn-text: #222;
-fx-theme-extend-text: #369658;
-fx-theme-links: #9c0084;
-fx-theme-breadcrums: #9c0084;
-fx-theme-ok: #00ff00;
-fx-theme-on: #00d97e;
-fx-theme-attention: #ff0000;
-fx-theme-warning: orange;
-fx-theme-title-label: #336699;
-fx-theme-field-label: #9c0084;
-fx-theme-breakpoint: #ff0000;
-fx-theme-breakpoint-continue: #369658;
-fx-theme-form-highlight: #BBBBBB;
-fx-theme-list-selection: #afbbc4;
-fx-theme-hl-combos: #b9b8d4;
-fx-theme-important-combos: #c9a9c4;
}
.root {
-fx-base: -fx-theme-base1;
-fx-background: -fx-theme-base1;
-fx-control-inner-background: -fx-theme-base1;
}
================================================
FILE: scripts/flow-clj
================================================
# Make a babashka script or shell script to run trampoline like clj -x com.foo/fn args
# clj -X:dbg:inst:dev:build flow-storm.api/trampoline :ns-set '#{\"hf.depstar\"}' :fn-symb 'hf.depstar/jar' :fn-args '[{:jar \"flow-storm-dbg.jar\" :aliases [:dbg] :paths-only false :sync-pom true :version \"1.1.1\" :group-id \"jpmonettas\" :artifact-id \"flow-storm-dbg\"}]'
================================================
FILE: scripts/gsettings
================================================
#!/home/jmonetta/bin/bb
(ns gsettings)
(require '[clojure.tools.cli :refer [parse-opts]])
(let [[first-arg] *command-line-args*
get-theme (fn [] (slurp "/home/jmonetta/.flow-storm/mock-current-gtk-theme"))]
(cond
(= first-arg "get")
(println (get-theme))
(= first-arg "monitor")
(loop [curr-theme (get-theme)
last-reported-theme nil]
(if (not= curr-theme last-reported-theme)
(do
(println (format "xxx %s" curr-theme))
(recur (get-theme) curr-theme))
(do
(Thread/sleep 1000)
(recur (get-theme) last-reported-theme))))))
================================================
FILE: scripts/mock-gnome.sh
================================================
#/bin/bash
export XDG_CURRENT_DESKTOP=gnome
export PATH=/home/jmonetta/my-projects/flow-storm-debugger/scripts/:$PATH
bash
================================================
FILE: shadow-cljs.edn
================================================
;; shadow-cljs configuration
{:deps {:aliases [:cljs-storm :dev]}
:builds
{:dev-test {:target :node-script
:main dev/-main
:output-to "public/dev-test.js"}}
:nrepl {:port 9000
:middleware [refactor-nrepl.middleware/wrap-refactor
cider.nrepl/cider-middleware
cider.piggieback/wrap-cljs-repl
flow-storm.nrepl.middleware/wrap-flow-storm]}
}
================================================
FILE: src-dbg/flow_storm/debugger/docs.clj
================================================
(ns flow-storm.debugger.docs
(:require [flow-storm.state-management :refer [defstate]]
[flow-storm.utils :as utils]
[clojure.java.io :as io]
[clojure.edn :as edn]))
(declare start)
(declare stop)
(declare fn-docs)
(def docs-file-name "flow-docs.edn")
(def deprecated-docs-file-name "samples.edn")
(defn- classpath-uris [file-name]
(let [cl (.. Thread currentThread getContextClassLoader)]
(->> (enumeration-seq (.getResources cl file-name))
(map #(.toURI %)))))
(defn- read-deprecated-docs [files]
(let [fns-data (reduce (fn [r file]
(merge r (-> file
slurp
edn/read-string)))
{}
files)]
{:functions/data fns-data}))
(defn- read-docs [files]
(when (seq files)
(utils/log (format "Loading docs using %s files" (pr-str files))))
(reduce (fn [r file]
(merge-with merge r (-> file
slurp
edn/read-string)))
{}
files))
(defn read-classpath-docs []
(let [dev-docs-file (io/file docs-file-name) ;; just to make dev easier
docs-files (cond-> (classpath-uris docs-file-name)
(.exists dev-docs-file) (conj dev-docs-file))
docs-maps (read-docs docs-files)
deprecated-docs-maps (read-deprecated-docs (classpath-uris deprecated-docs-file-name))]
(merge-with merge docs-maps deprecated-docs-maps)))
(defstate fn-docs
:start (fn [_] (start))
:stop (fn [] (stop)))
(defn start []
(-> (read-classpath-docs)
:functions/data))
(defn stop []
nil)
================================================
FILE: src-dbg/flow_storm/debugger/events_processor.clj
================================================
(ns flow-storm.debugger.events-processor
"Processing events the debugger receives from the runtime"
(:require [flow-storm.debugger.ui.browser.screen :as browser-screen]
[flow-storm.debugger.ui.outputs.screen :as outputs-screen]
[flow-storm.debugger.ui.main :as ui-main]
[flow-storm.debugger.ui.flows.screen :as flows-screen]
[flow-storm.debugger.ui.flows.bookmarks :as bookmarks]
[flow-storm.debugger.ui.docs.screen :as docs-screen]
[flow-storm.debugger.ui.flows.general :as ui-general :refer [show-message]]
[flow-storm.debugger.ui.utils :as ui-utils]
[flow-storm.debugger.ui.data-windows.data-windows :as data-windows]
[flow-storm.utils :refer [log]]
[flow-storm.jobs :refer [timeline-updates-check-interval]]
[flow-storm.debugger.state :as dbg-state]))
(defn- vanilla-var-instrumented-event [{:keys [var-ns var-name]}]
(ui-utils/run-later
(browser-screen/add-to-instrumentation-list (browser-screen/make-inst-var var-ns var-name))
(ui-general/select-main-tools-tab "tool-browser")))
(defn- vanilla-var-uninstrumented-event [{:keys [var-ns var-name]}]
(ui-utils/run-later
(browser-screen/remove-from-instrumentation-list (browser-screen/make-inst-var var-ns var-name))
(ui-general/select-main-tools-tab "tool-browser")))
(defn- vanilla-namespace-instrumented-event [{:keys [ns-name]}]
(ui-utils/run-later
(browser-screen/add-to-instrumentation-list (browser-screen/make-inst-ns ns-name))
(ui-general/select-main-tools-tab "tool-browser")))
(defn- vanilla-namespace-uninstrumented-event [{:keys [ns-name]}]
(ui-utils/run-later
(browser-screen/remove-from-instrumentation-list (browser-screen/make-inst-ns ns-name))
(ui-general/select-main-tools-tab "tool-browser")))
(defn- storm-instrumentation-updated-event [data]
(ui-utils/run-later
(browser-screen/update-storm-instrumentation data)
(ui-general/select-main-tools-tab "tool-browser")))
(defn- tap-event [{:keys [value]}]
(ui-utils/run-later
(outputs-screen/add-tap-value value)))
(defn out-write-event [{:keys [msg]}]
(ui-utils/run-later
(outputs-screen/add-out-write msg)))
(defn err-write-event [{:keys [msg]}]
(ui-utils/run-later
(outputs-screen/add-err-write msg)))
(defn last-evals-update-event [{:keys [last-evals-refs]}]
(ui-utils/run-later
(outputs-screen/update-last-evals last-evals-refs)))
(defn- flow-created-event [flow-info]
(ui-utils/run-now
(ui-main/create-flow flow-info)))
(defn- flow-discarded-event [flow-info]
(ui-utils/run-now
(flows-screen/clear-debugger-flow (:flow-id flow-info))))
(defn- threads-updated-event [{:keys [flow-id flow-threads-info]}]
(ui-utils/run-now
(flows-screen/update-threads-list flow-id flow-threads-info)))
(defn- timeline-updated-event [{:keys [flow-id thread-id]}]
;; If there is a tab open for the thread already, update it
(when (dbg-state/get-thread flow-id thread-id)
(ui-utils/run-later
(if (:auto-update-ui? (dbg-state/debugger-config))
(let [start (System/currentTimeMillis)]
(flows-screen/update-outdated-thread-ui flow-id thread-id)
(when (> (- (System/currentTimeMillis) start)
timeline-updates-check-interval)
(log "WARNING: UI updates are slow, disabling automatic updates.")
(dbg-state/set-auto-update-ui false)))
(flows-screen/make-outdated-thread flow-id thread-id)))))
(defn- task-submitted-event [_]
(ui-main/set-task-cancel-btn-enable true))
(defn- task-finished-event [_]
(ui-main/set-task-cancel-btn-enable false))
(defn- task-failed-event [{:keys [message]}]
(ui-main/set-task-cancel-btn-enable false)
(ui-general/show-message message :error))
(defn- heap-info-update-event [ev-args-map]
(ui-main/update-heap-indicator ev-args-map))
(defn- goto-location-event [loc]
(ui-utils/run-now
(flows-screen/goto-location loc)))
(defn- show-doc-event [{:keys [var-symbol]}]
(ui-utils/run-now
(ui-general/select-main-tools-tab "tool-docs")
(docs-screen/show-doc var-symbol)))
(defn- break-installed-event [{:keys [fq-fn-symb]}]
(ui-utils/run-later
(browser-screen/add-to-instrumentation-list (browser-screen/make-inst-break fq-fn-symb))))
(defn- break-removed-event [{:keys [fq-fn-symb]}]
(ui-utils/run-later
(browser-screen/remove-from-instrumentation-list (browser-screen/make-inst-break fq-fn-symb))))
(defn- recording-updated-event [{:keys [recording?]}]
(flows-screen/set-recording-btn recording?))
(defn- multi-timeline-recording-updated-event [{:keys [recording?]}]
(flows-screen/set-multi-timeline-recording-btn recording?))
(defn- function-unwinded-event [{:keys [flow-id] :as unwind-data}]
(case (dbg-state/maybe-add-exception unwind-data)
:ex-limit-reached (show-message "Too many exceptions throwed, showing only the first ones" :warning)
(:ex-skipped :ex-limit-passed) nil
:ex-added (ui-utils/run-later
(flows-screen/add-exception-to-menu unwind-data)
;; the first time we encounter an exception, navigate to that location
(when (and (:auto-jump-on-exception? (dbg-state/debugger-config))
(= 1 (count (dbg-state/flow-exceptions flow-id))))
(flows-screen/goto-location unwind-data)))))
(defn expression-bookmark-event [{:keys [flow-id thread-id idx note source] :as bookmark-location}]
(ui-utils/run-later
(dbg-state/add-bookmark {:flow-id flow-id
:thread-id thread-id
:idx idx
:source (or source :bookmark.source/api)
:note note})
(bookmarks/update-bookmarks)
;; jump to the first mark, unless, we've already jumped to an exception
(when (and
(= 1 (->> (dbg-state/flow-bookmarks flow-id)
(filter (fn [{:keys [source]}]
(= source :bookmark.source/api)))
count))
(not (and
(:auto-jump-on-exception? (dbg-state/debugger-config))
(seq (dbg-state/flow-exceptions flow-id)))))
(flows-screen/goto-location bookmark-location))))
(defn data-window-push-val-data-event [{:keys [dw-id val-data extras]}]
(data-windows/push-val dw-id val-data extras))
(defn data-window-update-event [{:keys [dw-id data]}]
(data-windows/update-val dw-id data))
(defn process-event [[ev-type ev-args-map]]
(case ev-type
:vanilla-var-instrumented (vanilla-var-instrumented-event ev-args-map)
:vanilla-var-uninstrumented (vanilla-var-uninstrumented-event ev-args-map)
:vanilla-namespace-instrumented (vanilla-namespace-instrumented-event ev-args-map)
:vanilla-namespace-uninstrumented (vanilla-namespace-uninstrumented-event ev-args-map)
:storm-instrumentation-updated-event (storm-instrumentation-updated-event ev-args-map)
:flow-created (flow-created-event ev-args-map)
:flow-discarded (flow-discarded-event ev-args-map)
:threads-updated (threads-updated-event ev-args-map)
:timeline-updated (timeline-updated-event ev-args-map)
:tap (tap-event ev-args-map)
:out-write (out-write-event ev-args-map)
:err-write (err-write-event ev-args-map)
:last-evals-update (last-evals-update-event ev-args-map)
:task-submitted (task-submitted-event ev-args-map)
:task-finished (task-finished-event ev-args-map)
:task-failed (task-failed-event ev-args-map)
:heap-info-update (heap-info-update-event ev-args-map)
:goto-location (goto-location-event ev-args-map)
:show-doc (show-doc-event ev-args-map)
:break-installed (break-installed-event ev-args-map)
:break-removed (break-removed-event ev-args-map)
:recording-updated (recording-updated-event ev-args-map)
:multi-timeline-recording-updated (multi-timeline-recording-updated-event ev-args-map)
:function-unwinded-event (function-unwinded-event ev-args-map)
:expression-bookmark-event (expression-bookmark-event ev-args-map)
:data-window-push-val-data (data-window-push-val-data-event ev-args-map)
:data-window-update (data-window-update-event ev-args-map)
nil ;; events-processor doesn't handle every event, specially tasks processing
))
================================================
FILE: src-dbg/flow_storm/debugger/events_queue.clj
================================================
(ns flow-storm.debugger.events-queue
"Namespace for the sub-component that manages an events queue.
This events are pushed by the runtime part (where the recordings live) via
direct calling in the local mode or via websockets in remote mode for letting
us (the debugger) know about interesting events on the runtime part of the world.
Events will be dispatched after a dispatch-fn is set with `set-dispatch-fn`"
(:require [flow-storm.state-management :refer [defstate]]
[flow-storm.debugger.state :as dbg-state]
[flow-storm.utils :as utils :refer [log]])
(:import [java.util.concurrent ArrayBlockingQueue TimeUnit]))
(declare start-events-queue)
(declare stop-events-queue)
(declare events-queue)
(defstate events-queue
:start (fn [_] (start-events-queue))
:stop (fn [] (stop-events-queue)))
(def queue-poll-interval 500)
(def wait-for-system-started-interval 1000)
(defn enqueue-event! [e]
(when-let [queue (:queue events-queue)]
(.put ^ArrayBlockingQueue queue e)))
(defn add-dispatch-fn [fn-key dispatch-fn]
(swap! (:dispatch-fns events-queue) assoc fn-key dispatch-fn))
(defn rm-dispatch-fn [fn-key]
(swap! (:dispatch-fns events-queue) dissoc fn-key))
(defn start-events-queue []
(let [events-queue (ArrayBlockingQueue. 1000)
dispatch-fns (atom {})
dispatch-thread (Thread.
(fn []
(try
;; don't do anything until we have a dispatch-fn
(while (and (not (.isInterrupted (Thread/currentThread)))
(empty? @dispatch-fns))
(utils/log "Waiting for a dispatch-fn before dispatching events")
(Thread/sleep wait-for-system-started-interval))
;; start the dispatch loop
(loop [ev nil]
(when-not (.isInterrupted (Thread/currentThread))
(try
(when ev
(when (and (:debug-mode? (dbg-state/debugger-config))
(not (= (first ev) :heap-info-update)))
(log (format "Processing event: %s" ev)))
(doseq [dispatch (vals @dispatch-fns)]
(dispatch ev)))
(catch Exception e
(utils/log-error (str "Error dispatching event" ev) e)))
(recur (.poll events-queue queue-poll-interval TimeUnit/MILLISECONDS))))
(catch java.lang.InterruptedException _
(utils/log "Events thread interrupted"))
(catch Exception e
(utils/log-error "Events queue thread error" e))))
"FlowStorm Events Processor")
interrupt-fn (fn [] (.interrupt dispatch-thread))]
(.start dispatch-thread)
{:interrupt-fn interrupt-fn
:queue events-queue
:dispatch-fns dispatch-fns}))
(defn stop-events-queue []
(when-let [stop-fn (:interrupt-fn events-queue)]
(stop-fn)))
================================================
FILE: src-dbg/flow_storm/debugger/main.clj
================================================
(ns flow-storm.debugger.main
" This is the main namespace for the debugger itself, the graphical part of FlowStorm,
being `start-debugger` the main entry point.
The debugger system is made of varios stateful components orchestrated by a custom state system
defined `flow-storm.state-management`, which is similar to `mount`.
We are choosing a custom state system so we depend on as less libraries as we can
to avoid version conflicts with users libraries.
The debugger will start a different set of components depending if we are running a
remote or local debugger.
The debugger is a local debugger when it runs in the same process as the debuggee and can call
all functions locally in contrast with remote debugging like when connecting to a ClojureScript system
or a remote Clojure system where calls had to be made via websocket and repl connections.
Look at each component namespace doc string for more details on them.
"
(:require [flow-storm.debugger.ui.main :as ui-main]
[flow-storm.debugger.state :as dbg-state]
[flow-storm.debugger.events-queue :as events-queue]
[flow-storm.debugger.events-processor :as events-processor]
[flow-storm.debugger.docs]
[flow-storm.debugger.runtime-api]
[flow-storm.debugger.websocket]
[flow-storm.debugger.repl.core :as repl-core]
[flow-storm.utils :as utils]
[flow-storm.state-management :as state-management]
[flow-storm.debugger.ui.utils :as ui-utils]
[flow-storm.debugger.ui.plugins :as plugins]
[clojure.string :as str]))
(def flow-storm-core-ns 'flow-storm.core)
(def local-debugger-state-vars
"References to sub-components for local debugger sessions"
[#'flow-storm.debugger.events-queue/events-queue
#'flow-storm.debugger.docs/fn-docs
#'flow-storm.debugger.state/state
#'flow-storm.debugger.ui.main/ui
#'flow-storm.debugger.runtime-api/rt-api])
(def remote-debugger-state-vars
"Extends `local-debugger-state-vars` with extra sub-components
for remote debugger sessions"
(into local-debugger-state-vars
[#'flow-storm.debugger.websocket/websocket-server
#'flow-storm.debugger.repl.core/repl]))
(defn stop-debugger
"Gracefully stop the debugger. Useful for reloaded workflows."
[]
(state-management/stop {}))
(defn debugger-config []
(let [theme-prop (System/getProperty "flowstorm.theme")
title-prop (System/getProperty "flowstorm.title")
styles-prop (System/getProperty "flowstorm.styles")
auto-update-ui-prop (System/getProperty "flowstorm.autoUpdateUI")
call-tree-update-prop (System/getProperty "flowstorm.callTreeUpdate")
ui-timeout-millis (System/getProperty "flowstorm.uiTimeoutMillis")
plugins-nss-set (->> (reduce-kv (fn [acc prop value]
(if (str/starts-with? prop "flowstorm.plugins.namespaces")
(into acc (str/split value #","))
acc))
#{}
(System/getProperties)))]
(cond-> {}
theme-prop (assoc :theme (keyword theme-prop))
styles-prop (assoc :styles styles-prop)
title-prop (assoc :title title-prop)
auto-update-ui-prop (assoc :auto-update-ui? (= "true" auto-update-ui-prop))
call-tree-update-prop (assoc :call-tree-update? (= "true" call-tree-update-prop))
ui-timeout-millis (assoc :ui-timeout-millis (Long/parseLong ui-timeout-millis))
(seq plugins-nss-set) (assoc :plugins-namespaces-set plugins-nss-set))))
(defn start-debugger
"Run the debugger.
`config` should be a map containing :
- `:local?` when false will also start a websocket server and listen for connections
- `:theme` can be one of `:light`, `:dark` or `:auto`
- `:styles` a string path to a css file if you want to override some started debugger styles
When `:local?` is false you can also provide `:runtime-host` `:debugger-host` and `:port` for the nrepl server.
`:runtime-host` should be the ip of the debuggee (defaults to localhost)
`:debugger-host` shoud be the ip where the debugger is running, since the debuggee needs to connect back to it (defaults to localhost)
`:pre-require` can be used with a symbol for requiring a ns before starting"
[{:keys [local? pre-require] :as config}]
(when pre-require (require pre-require))
;; Ensure a task bar icon is shown on MacOS.
(System/setProperty "apple.awt.UIElement" "false")
;; Initialize the JavaFX toolkit
(ui-utils/init-toolkit)
(let [config (-> config
(update :port (fn [port]
(or port
(some-> (try (slurp ".nrepl-port") (catch Exception _))
Integer/parseInt))))
(merge (debugger-config)))]
(plugins/load-plugins-namespaces config)
(if local?
;; start components for local debugging
(do
(state-management/start {:only local-debugger-state-vars
:config config})
(ui-main/setup-ui-from-runtime-config)
(ui-main/setup-instrumentation-ui))
;; else, start components for remote debugging
(let [ws-connected? (promise)
repl-connected? (promise)
fully-started (atom false)
signal-ws-connected (fn [conn?]
(ui-main/set-conn-status-lbl :ws conn?)
(dbg-state/set-connection-status :ws conn?))
signal-repl-connected (fn [conn?]
(dbg-state/set-connection-status :repl conn?)
(ui-main/set-conn-status-lbl :repl conn?))]
(state-management/start {:only remote-debugger-state-vars
:config (assoc config
:on-ws-event events-queue/enqueue-event!
:on-ws-down (fn []
(utils/log "WebSocket connection went away")
(signal-ws-connected false)
(ui-main/clear-ui))
:on-ws-up (fn [_]
(signal-ws-connected true)
(deliver ws-connected? true)
;; This is kind of hacky but will handle the ClojureScript page reload situation.
;; After a page reload all the runtime part has been restarted, so
;; we can re-init it through the repl and also re-setup the ui with whatever the
;; runtime contains in terms of settings.
;; But we need to skip this the first time the ws connection comes up
;; since maybe the system ins't fully started yet, we maybe don't even have a UI
(when @fully-started
(when (dbg-state/repl-config)
(repl-core/init-repl (dbg-state/env-kind)))
(ui-main/setup-ui-from-runtime-config)))
:on-repl-down (fn []
(signal-repl-connected false))
:on-repl-up (fn []
(deliver repl-connected? true)
(signal-repl-connected true)))})
(reset! fully-started true)
;; if there is a repl config wait for the connection before moving on
(when (and (dbg-state/repl-config)
@repl-connected?)
(signal-repl-connected true))
;; once we have both the UI started and the runtime-connected
;; initialize the UI with the info retrieved from the runtime
(when @ws-connected?
(signal-ws-connected true)
(ui-main/setup-ui-from-runtime-config)
(ui-main/setup-instrumentation-ui)))))
;; for both, local and remote
;; we set the events dispatch-fn afater `state-management/start` returns because
;; we know the UI is ready to start processing events
(events-queue/add-dispatch-fn :events-processor events-processor/process-event)
)
================================================
FILE: src-dbg/flow_storm/debugger/repl/core.clj
================================================
(ns flow-storm.debugger.repl.core
"Stateful component that handles debugger connection to repls.
`start-repl` will be called at startup and will use the current repl
config in `state` to start a repl connection.
Depending on the repl configuration in `state` it can start Clojure and ClojureScript
repl connections.
It can do runtime initialization sequences with `init-repl` to prepare the runtime
part by executing some repl instructions.
The main functions after the repl is ready are :
- `safe-eval-code-str` for evaluating Clojure
- `safe-cljs-eval-code-str` for evaluating ClojureScript
"
(:require [flow-storm.state-management :refer [defstate]]
[flow-storm.debugger.repl.nrepl :as nrepl]
[flow-storm.debugger.websocket]
[flow-storm.debugger.state :as dbg-state]
[flow-storm.utils :as utils]
[clojure.java.io :as io])
(:import [java.io OutputStream]))
(declare start-repl)
(declare stop-repl)
(declare init-repl)
(def repl-watchdog-interval 3000)
(def log-file-path "./repl-client-debug")
(declare repl)
(defstate repl
:start (fn [config] (start-repl config))
:stop (fn [] (stop-repl)))
(defn default-repl-ns []
(let [env-kind (dbg-state/env-kind)]
(case env-kind :clj "user" :cljs "shadow.user")))
(defn eval-code-str
"Evaluate `code-str` in the connected Clojure repl.
Will throw if anything goes wrong.
`ns` can be used to customize what namespace `code-str`should be executed in.
Returns the result object."
([code-str] (eval-code-str code-str nil))
([code-str ns]
(if-not (:repl-ready? (dbg-state/connection-status))
(utils/log-error "No repl available. You need a repl connection to use this functionality. Checkout the user guide.")
(let [ns (or ns (default-repl-ns))]
(when-let [repl-eval (:repl-eval repl)]
(try
(repl-eval code-str ns)
(catch clojure.lang.ExceptionInfo ei
(throw (ex-info "Error evaluating code on :clj repl"
(assoc (ex-data ei)
:code-str code-str
:ns ns))))))))))
(defn safe-eval-code-str
"Wrapper of `eval-code-str` that will not throw. Will just log
an error in case of exception."
[& args]
(try
(apply eval-code-str args)
(catch Exception e (utils/log-error (.getMessage e) e))))
(defn safe-cljs-eval-code-str
"Eval `code-str` in the clojurescript repl through the connected
clojure repl.
`ns` can be used to customize what namespace `code-str`should be executed in.
In case anything goes wrong it will not throw, just log an error.
Returns the result object. If it is a js object it will return a string."
([code-str] (safe-cljs-eval-code-str code-str nil))
([code-str ns]
(let [ns (or ns "cljs.user")]
(if-let [repl-eval-cljs (:repl-eval-cljs repl)]
(try
(repl-eval-cljs code-str ns)
(catch Exception e
(utils/log-error (.getMessage e) e)
(throw (ex-info "Error evaluating code on :cljs repl"
{:code-str code-str
:ns ns
:cause e}))))
(utils/log-error "No cljs repl available")))))
(defn make-cljs-repl-init-sequence []
[{:code "(do (in-ns 'shadow.user) nil)" :ns nil :repl-kind :clj}
{:code "(require '[flow-storm.api :include-macros true])" :ns nil :repl-kind :clj}
{:code "(require '[flow-storm.runtime.debuggers-api :include-macros true])" :ns "cljs.user" :repl-kind :cljs}])
(defn make-clj-repl-init-sequence []
(let [opts (select-keys (dbg-state/debugger-config) [:debugger-host :debugger-ws-port])]
[{:code "(do (in-ns 'user) nil)" :ns nil :repl-kind :clj}
{:code "(require '[flow-storm.api])" :ns "user" :repl-kind :clj}
{:code (format "(flow-storm.runtime.debuggers-api/remote-connect %s)" (pr-str opts)) :ns "user" :repl-kind :clj}]))
(defn init-repl
([env-kind] (init-repl env-kind (:repl-eval repl) (:repl-eval-cljs repl)))
([env-kind repl-eval repl-eval-cljs]
(let [repl-init-sequence (case env-kind
:clj (make-clj-repl-init-sequence)
:cljs (make-cljs-repl-init-sequence))]
(doseq [{:keys [code ns repl-kind]} repl-init-sequence]
(case repl-kind
:clj (repl-eval code ns)
:cljs (repl-eval-cljs code ns))))))
(defn- connect-and-init [{:keys [repl-type runtime-host port build-id on-repl-up]}]
(let [runtime-host (or runtime-host "localhost")
env-kind (if (#{:shadow} repl-type) :cljs :clj) ;; HACKY, this logic is replicated in `state`
repl-kind :nrepl ;; HACKY, this logic is replicated in `state`
log-file (io/file log-file-path)
^OutputStream log-output-stream (io/make-output-stream log-file {:append true
:encoding "UTF-8"})
connect (fn [] (case repl-kind
:nrepl (nrepl/connect runtime-host port)))
;; repl here will be a map with :repl-eval (fn [code-str ns] ) and :close-connection (fn [])
;; :repl-eval fn will eval on the specific repl and return the value always as a string
{:keys [repl-eval close-connection]} (connect)
eval-clj (fn [code-str ns]
(when-not (= code-str ":watch-dog-ping")
(.write log-output-stream (.getBytes (format "\n\n---- [ %s ] ---->\n" ns)))
(.write log-output-stream (.getBytes (pr-str code-str)))
(.flush log-output-stream))
(let [response (repl-eval code-str ns)]
(when-not (= code-str ":watch-dog-ping")
(.write log-output-stream (.getBytes "\n<---------\n"))
(.write log-output-stream (.getBytes (pr-str response)))
(.flush log-output-stream))
(try
(when response (read-string {} response))
(catch Exception e
(utils/log-error (format "Error reading the response %s. CAUSE : %s"
response
(.getMessage e)))))))
eval-cljs (fn [code-str ns]
(eval-clj (format "((requiring-resolve 'hansel.instrument.utils/eval-in-ns-fn-cljs) '%s '%s %s)"
ns
code-str
(pr-str {:build-id build-id}))
"shadow.user"))
repl-comp {:repl-eval eval-clj
:repl-eval-cljs eval-cljs
:close-connection (fn []
(close-connection)
(.close log-output-stream))}]
(utils/log "Initializing repl...")
(try
(init-repl env-kind eval-clj eval-cljs)
(catch Exception e
(utils/log-error "There was a problem initializing the remote runtime via repl" e)))
(let [repl-ok? (= :watch-dog-ping (eval-clj ":watch-dog-ping" "user"))]
(utils/log (str "Repl ok? : " repl-ok?))
(when repl-ok?
(on-repl-up)))
repl-comp))
(defn- watchdog-loop [{:keys [on-repl-down] :as config}]
(let [repl-watchdog-thread
(Thread.
(fn []
(utils/log "Starting the repl watchdog loop")
(try
(loop []
(let [repl-ok? (try
(= :watch-dog-ping (eval-code-str ":watch-dog-ping" "user"))
(catch clojure.lang.ExceptionInfo ei
(let [{:keys [error/type] :as exd} (ex-data ei)]
(utils/log (format "[WATCHDOG] error executing ping. %s %s" type exd))
;; just return that the repl is not ok when we got a socket-exception from
;; the underlying repl
(not= type :repl/socket-exception)))
(catch Exception e
(utils/log-error "This is completely unexpected :")
(.printStackTrace e)
false))]
(when-not repl-ok?
(utils/log "[WATCHDOG] repl looks down, trying to reconnect ...")
(on-repl-down)
(stop-repl)
(try
(let [new-repl-cmp (connect-and-init config)]
(alter-var-root #'repl (constantly new-repl-cmp)))
(catch Exception e
(utils/log (format "Couldn't restart repl (%s), retrying in %d ms" (.getMessage e) repl-watchdog-interval)))))
(Thread/sleep repl-watchdog-interval))
(recur))
(catch java.lang.InterruptedException _
(utils/log "FlowStorm Repl Watchdog thread interrupted"))
(catch Exception e (.printStackTrace e)))))
repl-watchdog-interrupt (fn [] (.interrupt repl-watchdog-thread))]
(.setName repl-watchdog-thread "FlowStorm Repl Watchdog")
(.start repl-watchdog-thread)
repl-watchdog-interrupt))
(defn start-repl [{:keys [port] :as config}]
(when port
(let [repl-comp (connect-and-init config)]
(watchdog-loop config)
repl-comp)))
(defn stop-repl []
(when-let [close-conn (:close-connection repl)]
(close-conn)))
================================================
FILE: src-dbg/flow_storm/debugger/repl/nrepl.clj
================================================
(ns flow-storm.debugger.repl.nrepl
"Utilities to connect to nRepl servers"
(:require [nrepl.core :as nrepl]
[nrepl.transport :as transport]
[flow-storm.utils :refer [log]]))
(defn connect
"Given `host` and `port` connects to a nRepl server.
Returns a map with two functions {:repl-eval , :close-connection}.
:repl-eval is a function of code and ns, both strings"
[host port]
(let [transport (nrepl/connect :host host
:port port
:transport-fn #'transport/bencode)
;; non of our commands should take longer than 10secs to execute
;; if more, we consider it a repl timeout and give up
client (nrepl/client transport 10000)
session (nrepl/client-session client)]
{:repl-eval (fn repl-eval [code-str ns]
(try
(let [msg (cond-> {:op "eval" :code code-str}
ns (assoc :ns ns))
responses (nrepl/message session msg)
{:keys [err] :as res-map} (nrepl/combine-responses responses)]
(if (empty? responses)
(throw (ex-info "nrepl timeout" {:error/type :repl/timeout}))
(if err
(throw (ex-info (str "nrepl evaluation error: " err)
(assoc res-map
:error/type :repl/evaluation-error
:msg msg)))
(first (:value res-map)))))
(catch java.net.SocketException se
(throw (ex-info (.getMessage se)
{:error/type :repl/socket-exception})))
(catch Exception e
(log (format "Error evaluating %s in NS %s, CAUSE: %s" code-str ns (.getMessage e))))))
:close-connection (fn []
(.close transport))}))
(comment
(def transport (nrepl/connect :host "localhost"
:port 9000
:transport-fn #'transport/bencode))
(def client (nrepl/client transport Long/MAX_VALUE))
(def client (nrepl/client transport 3000))
(def session (nrepl/client-session client))
(def res (nrepl/message session {:op "eval" :code "(require '[some.crazy :as c])"}))
(def res (nrepl/message session {:op "eval" :code "(do (require '[shadow.cljs.devtools.api :as shadow]) (shadow/nrepl-select :browser-repl))"}))
(def res (nrepl/message session {:op "describe"}))
(def res (nrepl/message session {:op "ls-sessions"}))
(def res (nrepl/message session {:op "eval" :code "(+ 1 2)"}))
(def res (nrepl/message session {:op "eval" :code "(in-ns 'user)"}))
(def res (nrepl/message session {:op "eval" :code "(do (require '[shadow.cljs.devtools.api :as shadow]) (shadow/watch :app) (shadow/nrepl-select :app))"}))
(def res (nrepl/message session {:op "eval" :code "js/window"}))
(def res (nrepl/message session {:op "eval" :code "a" :ns "cljs.user"}))
)
================================================
FILE: src-dbg/flow_storm/debugger/runtime_api.clj
================================================
(ns flow-storm.debugger.runtime-api
"Component that implements the api that the debugger
uses to call the runtime.
All debugger functionality is implemented agains this API.
The api is declared as a protocol `RuntimeApiP` and has two possible
instantiations :
- `LocalRuntimeApi` directly call functions, since we are on the same process
- `RemoteRuntimeApi` call funcitons through a websocket and a repl
All this is implemented runtime part in `flow-storm.runtime.debuggers-api` which is
the interface exposed by the runtime to debuggers."
(:require [flow-storm.state-management :refer [defstate]]
[flow-storm.utils :as utils :refer [log log-error]]
[flow-storm.debugger.repl.core :refer [safe-eval-code-str safe-cljs-eval-code-str stop-repl]]
[flow-storm.runtime.debuggers-api :as dbg-api]
[flow-storm.debugger.websocket :as websocket]
[flow-storm.debugger.state :as dbg-state]
[flow-storm.debugger.ui.flows.general :refer [show-message]]
[clojure.string :as str])
(:import [java.io Closeable]))
(declare ->LocalRuntimeApi)
(declare ->RemoteRuntimeApi)
(declare rt-api)
(declare api-call)
(def ^:dynamic *cache-disabled?* false)
(defstate rt-api
:start (fn [{:keys [local?]}]
(let [api-cache (atom {})]
(if local?
(->LocalRuntimeApi api-cache)
(->RemoteRuntimeApi api-cache))))
:stop (fn []
(when-let [cc (:close-connection rt-api)]
(cc))))
(defprotocol RuntimeApiP
(runtime-config [_])
(val-pprint [_ v opts])
(data-window-push-val-data [_ dw-id val-ref extra])
(get-form [_ form-id])
(timeline-count [_ flow-id thread-id])
(timeline-entry [_ flow-id thread-id idx drift])
(multi-thread-timeline-count [_ flow-id])
(frame-data [_ flow-id thread-id idx opts])
(bindings [_ flow-id thread-id idx opts])
(callstack-tree-root-node [_ flow-id thread-id])
(callstack-node-childs [_ node])
(callstack-node-frame [_ node])
(fn-call-stats [_ flow-id thread-id])
(interrupt-all-tasks [_])
(start-task [_ task-id])
(collect-fn-frames-task [_ flow-id thread-id fn-ns fn-name form-id render-args render-ret?])
(find-expr-entry-task [_ criteria])
(total-order-timeline-task [_ opts])
(thread-prints-task [_ print-cfg])
(search-collect-timelines-entries-task [_ criteria opts])
(discard-flow [_ flow-id])
(def-value [_ var-symb val-ref])
(tap-value [_ v])
(get-all-namespaces [_])
(get-all-vars-for-ns [_ nsname])
(get-var-meta [_ var-ns var-name])
(vanilla-instrument-var [_ var-ns var-name opts])
(vanilla-uninstrument-var [_ var-ns var-name opts])
(vanilla-instrument-namespaces [_ nsnames opts])
(vanilla-uninstrument-namespaces [_ nanames opts])
(modify-storm-instrumentation [_ operation opts])
(get-storm-instrumentation [_])
(storm-instrumentation-enable? [_])
(turn-storm-instrumentation [_ on?])
(reload-namespace [_ ns-info])
(eval-form [_ form-str opts])
(clear-runtime-state [_])
(clear-api-cache [_])
(clear-outputs [_])
(all-flows-threads [_])
(flow-threads-info [_ flow-id])
(unblock-thread [_ thread-id])
(unblock-all-threads [_])
(add-breakpoint [_ fq-fn-symb opts])
(remove-breakpoint [_ fq-fn-symb opts])
(stack-for-frame [_ flow-id thread-id fn-call-idx])
(toggle-recording [_])
(toggle-multi-timeline-recording [_])
(switch-record-to-flow [_ flow-id])
(find-fn-call-task [_ fq-fn-call-symb from-idx opts])
(set-thread-trace-limit [_ limit])
(set-heap-limit [_ limit])
(call-by-fn-key [_ fn-key args-vec]))
(defn cached-apply [cache cache-key f args]
(let [res (get @cache cache-key :flow-storm/cache-miss)]
(if (or *cache-disabled?*
(= res :flow-storm/cache-miss))
;; miss or disabled, we need to call
(let [new-res (apply f args)]
(when (:debug-mode? (dbg-state/debugger-config)) (log (utils/colored-string "CALLED" :red)))
(swap! cache assoc cache-key new-res)
new-res)
;; hit, return cached
(do
(when (:debug-mode? (dbg-state/debugger-config)) (log "CACHED"))
res))))
(defn api-call
"Calling the runtime side function fkey with args. The call-type can be :local or :remote.
If a callback is provided then this function will be async, nil will be returned and
the callback will be called with the response. Otherwise it will block and return the response."
([call-type fkey args] (api-call call-type fkey args {} nil))
([call-type fkey args opts] (api-call call-type fkey args opts nil))
([call-type fkey args {:keys [cache]} callback]
(let [timeout (:ui-timeout-millis (dbg-state/debugger-config))
f (case call-type
:local (dbg-api/api-fn-by-key fkey)
:remote (fn [& args] (websocket/sync-remote-api-request fkey args)))
debug-mode? (:debug-mode? (dbg-state/debugger-config))]
(when debug-mode? (log (format "%s API-CALL %s %s" call-type fkey (pr-str args))))
;; make the calls in a future so we can have a timeout and don't block the UI thread
;; forever
(let [call-resp-fut (future
(let [result (if cache
(let [cache-key (into [fkey] args)]
(cached-apply cache cache-key f args))
(do
(when debug-mode? (log (utils/colored-string "CALLED" :red)))
(apply f args)))]
(if callback
(callback result)
result)))]
(when-not callback
(let [call-resp (deref call-resp-fut timeout :flow-storm/call-time-out)]
(if (= call-resp :flow-storm/call-time-out)
(let [msg (format "A call to %s timed out" fkey)]
(show-message msg :warning)
(throw (ex-info msg {:call-type call-type :fkey fkey :args args})))
call-resp)))))))
(defn- config-dw-extras [extras]
(assoc extras
:pprint-previews? (:pprint-previews? (dbg-state/debugger-config))))
(defrecord LocalRuntimeApi [api-cache]
RuntimeApiP
(runtime-config [_] (api-call :local :runtime-config []))
(val-pprint [_ v opts] (api-call :local :val-pprint [v opts] {:cache api-cache})) ;; CACHED
(data-window-push-val-data [_ dw-id val-ref extra] (api-call :local :data-window-push-val-data [dw-id val-ref (config-dw-extras extra)]))
(get-form [_ form-id] (api-call :local :get-form [form-id] {:cache api-cache})) ;; CACHED
(timeline-count [_ flow-id thread-id] (api-call :local :timeline-count [flow-id thread-id]))
(timeline-entry [_ flow-id thread-id idx drift] (api-call :local :timeline-entry [flow-id thread-id idx drift]))
(multi-thread-timeline-count [_ flow-id] (api-call :local :multi-thread-timeline-count [flow-id]))
(frame-data [_ flow-id thread-id idx opts] (api-call :local :frame-data [flow-id thread-id idx opts]))
(bindings [_ flow-id thread-id idx opts] (api-call :local :bindings [flow-id thread-id idx opts]))
(callstack-tree-root-node [_ flow-id thread-id] (api-call :local :callstack-tree-root-node [flow-id thread-id]))
(callstack-node-childs [_ node] (api-call :local :callstack-node-childs [node]))
(callstack-node-frame [_ node] (api-call :local :callstack-node-frame [node]))
(fn-call-stats [_ flow-id thread-id] (api-call :local :fn-call-stats [flow-id thread-id]))
(collect-fn-frames-task [_ flow-id thread-id fn-ns fn-name form-id render-args render-ret?] (api-call :local :collect-fn-frames-task [flow-id thread-id fn-ns fn-name form-id render-args render-ret?]))
(start-task [_ task-id] (api-call :local :start-task [task-id]))
(interrupt-all-tasks [_] (api-call :local :interrupt-all-tasks []))
(find-expr-entry-task [_ criteria] (api-call :local :find-expr-entry-task [criteria]))
(total-order-timeline-task [_ opts] (api-call :local :total-order-timeline-task [opts]))
(thread-prints-task [_ print-cfg] (api-call :local :thread-prints-task [print-cfg]))
(search-collect-timelines-entries-task [_ criteria opts] (api-call :local :search-collect-timelines-entries-task [criteria opts]))
(discard-flow [_ flow-id] (api-call :local :discard-flow [flow-id]))
(def-value [_ var-symb val-ref] (api-call :local :def-value [(or (namespace var-symb) "user") (name var-symb) val-ref]))
(tap-value [_ vref] (api-call :local :tap-value [vref]))
(get-all-namespaces [_] (mapv (comp str ns-name) (all-ns)))
(get-all-vars-for-ns [_ nsname] (->> (ns-interns (symbol nsname)) keys (map str)))
(get-var-meta [_ var-ns var-name]
(-> (meta (resolve (symbol var-ns var-name)))
(update :ns (comp str ns-name))))
(vanilla-instrument-var [_ var-ns var-name opts]
(api-call :local :vanilla-instrument-var [:clj (symbol var-ns var-name) opts]))
(vanilla-uninstrument-var [_ var-ns var-name opts]
(api-call :local :vanilla-uninstrument-var [:clj (symbol var-ns var-name) opts]))
(vanilla-instrument-namespaces [_ nsnames {:keys [profile] :as opts}]
(let [disable-set (utils/disable-from-profile profile)]
(api-call :local :vanilla-instrument-namespaces [:clj nsnames (assoc opts :disable disable-set)])))
(vanilla-uninstrument-namespaces [_ nsnames opts]
(api-call :local :vanilla-uninstrument-namespaces [:clj nsnames opts]))
(modify-storm-instrumentation [_ operation opts]
(api-call :local :modify-storm-instrumentation [:clj operation opts]))
(get-storm-instrumentation [_]
(api-call :local :get-storm-instrumentation [:clj]))
(storm-instrumentation-enable? [_]
(api-call :local :storm-instrumentation-enable? [:clj]))
(turn-storm-instrumentation [_ on?]
(api-call :local :turn-storm-instrumentation [:clj on?]))
(reload-namespace [_ ns-info]
(require (symbol (:namespace-name ns-info)) :reload))
(eval-form [_ form-str {:keys [instrument? instrument-options var-name ns]}]
(let [ns-to-eval (find-ns (symbol ns))]
(binding [*ns* ns-to-eval]
(let [form (read-string
(if instrument?
(format "(flow-storm.api/instrument* %s %s)" (pr-str instrument-options) form-str)
form-str))
[v vmeta] (when var-name
(let [v (find-var (symbol ns var-name))]
[v (meta v)]))]
;; Don't eval the form in the same UI thread or
;; it will deadlock because the same thread can generate
;; a event that will be waiting for the UI thread
(.start
(Thread.
(fn []
(binding [*ns* ns-to-eval]
(try
(eval form)
(catch Exception ex
(log-error "Error instrumenting form" ex)
(if (and (.getCause ex) (str/includes? (.getMessage (.getCause ex)) "Method code too large!"))
(show-message "The form you are trying to instrument exceeds the method limit after instrumentation. Try intrumenting it without bindings." :error)
(show-message (.getMessage ex) :error))))
;; when we evaluate a function from the repl we lose all meta
;; so when re-evaluating a var (probably a function) store and restore its meta
(when v (reset-meta! v vmeta))))))))))
(clear-runtime-state [_]
(api-call :local :clear-runtime-state []))
(clear-api-cache [_]
(reset! api-cache {}))
(clear-outputs [_]
(api-call :local :clear-outputs []))
(flow-threads-info [_ flow-id]
(api-call :local :flow-threads-info [flow-id]))
(unblock-thread [_ thread-id]
(api-call :local :unblock-thread [thread-id]))
(unblock-all-threads [_]
(api-call :local :unblock-all-threads []))
(add-breakpoint [_ fq-fn-symb opts]
(api-call :local :add-breakpoint! [fq-fn-symb opts]))
(remove-breakpoint [_ fq-fn-symb opts]
(api-call :local :remove-breakpoint! [fq-fn-symb opts]))
(all-flows-threads [_]
(api-call :local :all-flows-threads []))
(stack-for-frame [_ flow-id thread-id fn-call-idx]
(api-call :local :stack-for-frame [flow-id thread-id fn-call-idx]))
(toggle-recording [_]
(api-call :local :toggle-recording []))
(toggle-multi-timeline-recording [_]
(api-call :local :toggle-multi-timeline-recording []))
(switch-record-to-flow [_ flow-id]
(api-call :local :switch-record-to-flow [flow-id]))
(find-fn-call-task [_ fq-fn-call-symb from-idx opts]
(api-call :local :find-fn-call-task [fq-fn-call-symb from-idx opts]))
(set-thread-trace-limit [_ limit]
(api-call :local :set-thread-trace-limit [limit]))
(set-heap-limit [_ limit]
(api-call :local :set-heap-limit [limit]))
(call-by-fn-key [_ fn-key args-vec]
(api-call :local fn-key args-vec)))
;;;;;;;;;;;;;;;;;;;;;;
;; For Clojure repl ;;
;;;;;;;;;;;;;;;;;;;;;;
(defrecord RemoteRuntimeApi [api-cache]
RuntimeApiP
(runtime-config [_] (api-call :remote :runtime-config []))
(val-pprint [_ v opts] (api-call :remote :val-pprint [v opts] {:cache api-cache})) ;; CACHED
(data-window-push-val-data [_ dw-id val-ref extra] (api-call :remote :data-window-push-val-data [dw-id val-ref (config-dw-extras extra)]))
(get-form [_ form-id] (api-call :remote :get-form [form-id] {:cache api-cache})) ;; CACHED
(timeline-count [_ flow-id thread-id] (api-call :remote :timeline-count [flow-id thread-id]))
(timeline-entry [_ flow-id thread-id idx drift] (api-call :remote :timeline-entry [flow-id thread-id idx drift]))
(multi-thread-timeline-count [_ flow-id] (api-call :remote :multi-thread-timeline-count [flow-id]))
(frame-data [_ flow-id thread-id idx opts] (api-call :remote :frame-data [flow-id thread-id idx opts]))
(bindings [_ flow-id thread-id idx opts] (api-call :remote :bindings [flow-id thread-id idx opts]))
(callstack-tree-root-node [_ flow-id thread-id] (api-call :remote :callstack-tree-root-node [flow-id thread-id]))
(callstack-node-childs [_ node] (api-call :remote :callstack-node-childs [node]))
(callstack-node-frame [_ node] (api-call :remote :callstack-node-frame [node]))
(fn-call-stats [_ flow-id thread-id] (api-call :remote :fn-call-stats [flow-id thread-id]))
(collect-fn-frames-task [_ flow-id thread-id fn-ns fn-name form-id render-args render-ret?] (api-call :remote :collect-fn-frames-task [flow-id thread-id fn-ns fn-name form-id render-args render-ret?]))
(start-task [_ task-id] (api-call :remote :start-task [task-id]))
(interrupt-all-tasks [_] (api-call :remote :interrupt-all-tasks []))
(find-expr-entry-task [_ criteria] (api-call :remote :find-expr-entry-task [criteria]))
(total-order-timeline-task [_ opts] (api-call :remote :total-order-timeline-task [opts]))
(thread-prints-task [_ print-cfg] (api-call :remote :thread-prints-task [print-cfg]))
(search-collect-timelines-entries-task [_ criteria opts] (api-call :remote :search-collect-timelines-entries-task [criteria opts]))
(discard-flow [_ flow-id] (api-call :remote :discard-flow [flow-id]))
(def-value [_ var-symb val-ref]
(case (dbg-state/env-kind)
:clj (api-call :remote :def-value [(or (namespace var-symb) "user") (name var-symb) val-ref])
:cljs (safe-cljs-eval-code-str (format "(def %s (flow-storm.runtime.values/deref-value (flow-storm.types/make-value-ref %d)))"
(name var-symb)
(:vid val-ref))
(or (namespace var-symb) "cljs.user"))))
(tap-value [_ vref] (api-call :remote :tap-value [vref]))
(get-all-namespaces [_]
(case (dbg-state/env-kind)
:clj (api-call :remote :all-namespaces [:clj])
:cljs (safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/all-namespaces :cljs %s)" (:repl.cljs/build-id (dbg-state/repl-config))))))
(get-all-vars-for-ns [_ nsname]
(case (dbg-state/env-kind)
:clj (api-call :remote :all-vars-for-namespace [:clj (symbol nsname)])
:cljs (safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/all-vars-for-namespace :cljs '%s %s)" nsname (:repl.cljs/build-id (dbg-state/repl-config))))))
(get-var-meta [_ var-ns var-name]
(case (dbg-state/env-kind)
:clj (api-call :remote :get-var-meta [:clj (symbol var-ns var-name)])
:cljs (safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/get-var-meta :cljs '%s/%s %s)" var-ns var-name {:build-id (:repl.cljs/build-id (dbg-state/repl-config))}))))
(vanilla-instrument-var [_ var-ns var-name opts]
(case (dbg-state/env-kind)
:clj (api-call :remote :vanilla-instrument-var [:clj (symbol var-ns var-name) opts])
:cljs (let [opts (assoc opts :build-id (:repl.cljs/build-id (dbg-state/repl-config)))]
(show-message "FlowStorm ClojureScript single var instrumentation is pretty limited. You can instrument them only once, and the only way of uninstrumenting them is by reloading your page or restarting your node process. Also deep instrumentation is missing some cases. So for most cases you are going to be better with [un]instrumenting entire namespaces." :warning)
(safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/vanilla-instrument-var :cljs '%s/%s %s)" var-ns var-name opts)))))
(vanilla-uninstrument-var [_ var-ns var-name opts]
(case (dbg-state/env-kind)
:clj (api-call :remote :vanilla-uninstrument-var [:clj (symbol var-ns var-name) opts])
:cljs (let [_opts (assoc opts :build-id (:repl.cljs/build-id (dbg-state/repl-config)))]
(show-message "FlowStorm currently can't uninstrument single vars in ClojureScript. You can only [un]instrument entire namespaces. If you want to get rid of the current vars instrumentation please reload your browser page, or restart your node process." :warning)
#_(safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/uninstrument-var :cljs '%s/%s %s)" var-ns var-name opts)))))
(vanilla-instrument-namespaces [_ nsnames {:keys [profile] :as opts}]
(let [opts (assoc opts :disable (utils/disable-from-profile profile))]
(case (dbg-state/env-kind)
:cljs (let [opts (assoc opts :build-id (:repl.cljs/build-id (dbg-state/repl-config)))]
(safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/vanilla-instrument-namespaces :cljs %s %s)" (into #{} nsnames) opts)))
:clj (api-call :remote :vanilla-instrument-namespaces [:clj (into #{} nsnames) opts]))))
(vanilla-uninstrument-namespaces [_ nsnames opts]
(case (dbg-state/env-kind)
:cljs (let [opts (assoc opts :build-id (:repl.cljs/build-id (dbg-state/repl-config)))]
(safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/vanilla-uninstrument-namespaces :cljs %s %s)" (into #{} nsnames) opts)))
;; for Clojure just call the api
:clj (api-call :remote :vanilla-uninstrument-namespaces [:clj (into #{} nsnames) opts])))
(modify-storm-instrumentation [_ operation opts]
(case (dbg-state/env-kind)
:cljs (let [opts (assoc opts :build-id (:repl.cljs/build-id (dbg-state/repl-config)))]
(safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/modify-storm-instrumentation :cljs %s %s)" operation opts)))
;; for Clojure just call the api
:clj (api-call :remote :modify-storm-instrumentation [:clj operation opts])))
(get-storm-instrumentation [_]
(case (dbg-state/env-kind)
:cljs (safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/get-storm-instrumentation :cljs)"))
;; for Clojure just call the api
:clj (api-call :remote :get-storm-instrumentation [:clj])))
(storm-instrumentation-enable? [_]
(case (dbg-state/env-kind)
:cljs (safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/storm-instrumentation-enable? :cljs)"))
;; for Clojure just call the api
:clj (api-call :remote :storm-instrumentation-enable? [:clj])))
(turn-storm-instrumentation [_ on?]
(case (dbg-state/env-kind)
:cljs (safe-eval-code-str (format "(flow-storm.runtime.debuggers-api/turn-storm-instrumentation :cljs %s)" on?))
;; for Clojure just call the api
:clj (api-call :remote :turn-storm-instrumentation [:clj on?])))
(reload-namespace [_ ns-info]
(let [reload-code (format "(require '%s :reload)" (:namespace-name ns-info))]
(case (dbg-state/env-kind)
:cljs (safe-cljs-eval-code-str reload-code "cljs.user")
:clj (safe-eval-code-str reload-code))))
(eval-form [this form-str {:keys [instrument? instrument-options var-name ns]}]
(let [var-meta (when var-name (select-keys (get-var-meta this ns var-name) [:file :column :end-column :line :end-line]))
form-expr (if instrument?
(format "(flow-storm.api/instrument* %s %s)" (pr-str instrument-options) form-str)
form-str)
expr-res (case (dbg-state/env-kind)
:clj (safe-eval-code-str form-expr ns)
:cljs (safe-cljs-eval-code-str form-expr ns) )]
(when (and (= :clj (dbg-state/env-kind))
var-meta)
;; for vars restore the meta attributes that get lost when we re eval from the repl
(safe-eval-code-str (format "(alter-meta! #'%s/%s merge %s)" ns var-name (pr-str var-meta))))
expr-res))
(clear-runtime-state [_]
(api-call :remote :clear-runtime-state []))
(clear-api-cache [_]
(reset! api-cache {}))
(clear-outputs [_]
(api-call :remote :clear-outputs []))
(flow-threads-info [_ flow-id]
(api-call :remote :flow-threads-info [flow-id]))
(unblock-thread [_ thread-id]
(api-call :remote :unblock-thread [thread-id]))
(unblock-all-threads [_]
(api-call :remote :unblock-all-threads []))
(add-breakpoint [_ fq-fn-symb opts]
(case (dbg-state/env-kind)
:clj (api-call :remote :add-breakpoint! [fq-fn-symb opts])
:cljs (show-message "Operation not supported for ClojureScript" :warning)))
(remove-breakpoint [_ fq-fn-symb opts]
(api-call :remote :remove-breakpoint! [fq-fn-symb opts]))
(all-flows-threads [_]
(api-call :remote :all-flows-threads []))
(stack-for-frame [_ flow-id thread-id fn-call-idx]
(api-call :remote :stack-for-frame [flow-id thread-id fn-call-idx]))
(toggle-recording [_]
(api-call :remote :toggle-recording []))
(toggle-multi-timeline-recording [_]
(api-call :remote :toggle-multi-timeline-recording []))
(switch-record-to-flow [_ flow-id]
(api-call :remote :switch-record-to-flow [flow-id]))
(find-fn-call-task [_ fq-fn-call-symb from-idx opts]
(api-call :remote :find-fn-call-task [fq-fn-call-symb from-idx opts]))
(set-thread-trace-limit [_ limit]
(api-call :remote :set-thread-trace-limit [limit]))
(set-heap-limit [_ limit]
(api-call :remote :set-heap-limit [limit]))
(call-by-fn-key [_ fn-key args-vec]
(api-call :remote fn-key args-vec))
Closeable
(close [_] (stop-repl))
)
================================================
FILE: src-dbg/flow_storm/debugger/state.clj
================================================
(ns flow-storm.debugger.state
"Sub component that manages the state of the debugger.
This is the state for supporting the UI, not the runtime part, where the
timelines are recorded.
All the state is inside one atom `state` which is specified by the `::state` spec."
(:require [flow-storm.state-management :refer [defstate]]
[flow-storm.utils :refer [pop-n] :as utils]
[flow-storm.debugger.ui.plugins :as plugins]
[clojure.java.io :as io]
[clojure.spec.alpha :as s])
(:import [javafx.stage Stage]))
(s/def ::timestamp (s/nilable int?))
(s/def ::print-level int?)
(s/def ::print-length int?)
(s/def :flow-storm/timeline-entry map?)
(s/def :flow-storm/val-id int?)
(s/def :flow-storm/val-ref record? #_(s/keys :req-un [:flow-storm/val-id]))
(s/def :flow-storm/fn-name string?)
(s/def :flow-storm/fn-ns string?)
(s/def :flow-storm/form-id int?)
(s/def :flow-storm/coord (s/coll-of int?))
(s/def :flow-storm.frame/ret :flow-storm/val-ref)
(s/def :flow-storm.frame/fn-call-idx int?)
(s/def :flow-storm.frame/parent-fn-call-idx (s/nilable int?))
(s/def :flow-storm.frame/args-vec any? #_(s/coll-of :flow-storm/val-ref)) ;; TODO: fix this https://clojure.atlassian.net/browse/CLJ-1975
(s/def :flow-storm.frame/expr-executions (s/coll-of :flow-storm/timeline-entry)) ;; TODO: this could be refined, since they can only by :expr and :fn-return
(s/def :return/kind #{:waiting :unwind :return})
(s/def :flow-storm/frame (s/keys :req [:return/kind]
:req-un [:flow-storm.frame/fn-call-idx
:flow-storm.frame/parent-fn-call-idx
:flow-storm/fn-name
:flow-storm/fn-ns
:flow-storm/form-id
:flow-storm.frame/args-vec
:flow-storm.frame/expr-executions]
:opt-un [:flow-storm.frame/ret
:flow-storm.frame/throwable]))
(s/def :thread/id int?)
(s/def :thread/name string?)
(s/def :thread/blocked (s/nilable (s/tuple :flow-storm/fn-ns :flow-storm/fn-name)))
(s/def :thread/curr-timeline-entry (s/nilable :flow-storm/timeline-entry))
(s/def :thread/curr-frame :flow-storm/frame)
(s/def :thread.ui.callstack-tree-hidden-fns/ref (s/keys :req-un [:flow-storm/fn-name
:flow-storm/fn-ns]))
(s/def :thread.ui/selected-functions-list-fn (s/nilable
(s/keys :req-un [:flow-storm/fn-name
:flow-storm/fn-ns
:flow-storm/form-id])))
(s/def :thread.ui/callstack-tree-hidden-fns (s/coll-of :thread.ui.callstack-tree-hidden-fns/ref))
(s/def :navigation-history/history (s/coll-of :flow-storm/timeline-entry))
(s/def :thread/navigation-history (s/keys :req-un [:navigation-history/head-pos
:navigation-history/history]))
(s/def :thread.exception/ex-type string?)
(s/def :thread.exception/ex-message string?)
(s/def :thread.exception/ex-hash int?)
(s/def :thread/exception (s/keys :req-un [:flow-storm/fn-name
:flow-storm/fn-ns
:thread.exception/ex-type
:thread.exception/ex-message
:thread.exception/ex-hash]))
(s/def :flow/exceptions (s/map-of :thread.exception/ex-hash :thread/exception))
(s/def :flow/exceptions-limit-reached? boolean?)
(s/def :flow/thread (s/keys :req [:thread/id
:thread/curr-timeline-entry
:thread/navigation-history]
:opt [:thread/curr-frame
:thread.ui/callstack-tree-hidden-fns
:thread.ui/selected-functions-list-fn]))
(s/def :flow/threads (s/map-of :thread/id :flow/thread))
(s/def :flow/id int?)
(s/def :flow/flow (s/keys :req [:flow/id
:flow/threads
:flow/exceptions
:flow/exceptions-limit-reached?]
:req-un [::timestamp]))
(s/def :flow/flows (s/map-of :flow/id :flow/flow))
(s/def :thread/info (s/keys :req [:flow/id
:thread/id
:thread/name]
:opt [:thread/blocked]))
(s/def :flow/threads-info (s/map-of :flow/id :thread/info))
(s/def :printer/enable? boolean?)
(s/def :printer/source-expr any?)
(s/def :printer/transform-expr-str string?)
(s/def :printer/printer (s/keys :req-un [:flow-storm/coord
::print-level
::print-length
:printer/enable?
:printer/transform-expr-str
:printer/source-expr]))
(s/def :printer/flow-printers (s/map-of :flow-storm/form-id
(s/map-of :flow-storm/coord
:printer/printer)))
(s/def :printer/printers (s/map-of :flow/id :printer/flow-printers))
(s/def :ui/selected-font-size-style-idx int?)
(s/def :ui/selected-theme #{:light :dark})
(s/def :ui/selected-tool keyword?)
(s/def :ui/extra-styles (s/nilable string?))
(s/def ::ws-ready? boolean?)
(s/def ::repl-ready? boolean?)
(s/def ::connection-status (s/keys :req-un [::ws-ready? ::repl-ready?]))
(s/def :ui.object/id string?)
(s/def :ui.object/node any?)
(s/def :ui.jfx-nodes-index/flow-id (s/nilable (s/or :flow-id :flow/id
:no-flow #{:no-flow})))
(s/def :ui/jfx-nodes-index (s/map-of (s/tuple :ui.jfx-nodes-index/flow-id
(s/nilable :thread/id)
:ui.object/id)
(s/coll-of :ui.object/node)))
(s/def ::jfx-stage #(instance? Stage %))
(s/def :ui/jfx-stages (s/coll-of ::jfx-stage))
(s/def :task/event-key keyword?)
(s/def :task/id any?)
(s/def ::pending-tasks-subscriptions (s/map-of (s/tuple :task/event-key :task/id)
fn?))
(s/def ::clojure-storm-env? boolean?)
(s/def ::local-mode? boolean?)
(s/def :config/env-kind #{:clj :cljs})
(s/def :config/storm? boolean?)
(s/def :config/flow-storm-nrepl-middleware? boolean?)
(s/def :status/recording? boolean?)
(s/def :status/total-order-recording? boolean?)
(s/def :status/breakpoints (s/coll-of (s/tuple :flow-storm/fn-ns :flow-storm/fn-name)))
(s/def ::runtime-config (s/nilable
(s/keys :req-un [:config/env-kind
:config/storm?
:config/flow-storm-nrepl-middleware?
:status/recording?
:status/total-order-recording?
:status/breakpoints])))
(s/def :repl/kind #{:nrepl})
(s/def :repl/type #{:shadow :clojure})
(s/def :repl/port int?)
(s/def :repl.cljs/build-id keyword?)
(s/def :config/repl (s/nilable
(s/keys :req [:repl/kind
:repl/type
:repl/port]
:opt [:repl.cljs/build-id])))
(s/def :config/debugger-host string?)
(s/def :config/debugger-ws-port int?)
(s/def :config/runtime-host string?)
(s/def :config/debug-mode? boolean?)
(s/def :config/auto-jump-on-exception? boolean?)
(s/def :config/auto-update-ui? boolean?)
(s/def :config/ui-timeout-millis number?)
(s/def :config/pprint-previews? boolean?)
(s/def ::debugger-config (s/keys :req-un [:config/repl
:config/debugger-host
:config/debugger-ws-port
:config/runtime-host
:config/debug-mode?
:config/pprint-previews?
:config/auto-jump-on-exception?
:config/auto-update-ui?
:config/ui-timeout-millis
:config/call-tree-update?]))
(s/def :bookmark/id (s/tuple :flow/id :thread/id int?))
(s/def :bookmark/flow-id :flow/id)
(s/def :bookmark/thread-id :thread/id)
(s/def :bookmark/idx int?)
(s/def :bookmark/note (s/nilable string?))
(s/def :bookmark/source #{:bookmark.source/ui
:bookmark.source/api})
(s/def ::bookmark (s/keys :req-un [:bookmark/flow-id
:bookmark/thread-id
:bookmark/idx
:bookmark/note
:bookmark/source]))
(s/def ::bookmarks (s/map-of :bookmark/id ::bookmark))
(s/def :data-window/breadcrums-box :ui.object/node)
(s/def :data-window/visualizers-combo-box :ui.object/node)
(s/def :data-window/val-box :ui.object/node)
(s/def :data-window/type-lbl :ui.object/node)
(s/def :visualizer/on-create ifn?)
(s/def :visualizer/on-update ifn?)
(s/def :visualizer/on-destroy ifn?)
(s/def :data-window.frame/val-data map?)
(s/def :data-window.frame/visualizer-combo :ui.object/node)
(s/def :data-window.frame/visualizer (s/keys :req-un [:visualizer/on-create]
:opt-un [:visualizer/on-update
:visualizer/on-destroy]))
(s/def :fx/node :ui.object/node)
(s/def :data-window.frame/visualizer-val-ctx (s/keys :req [:fx/node]))
(s/def :data-window/frame (s/keys :req-un [:data-window.frame/val-data
:data-window.frame/visualizer-combo
:data-window.frame/visualizer
:data-window.frame/visualizer-val-ctx]))
(s/def :data-window/stack (s/coll-of :data-window/frame))
(s/def :data-windows/data-window (s/keys :req-un [:data-window/breadcrums-box
:data-window/visualizers-combo-box
:data-window/val-box
:data-window/type-lbl
:data-window/stack]))
(s/def :data-window/id any?)
(s/def ::data-windows (s/map-of :data-window/id :data-windows/data-window))
(s/def ::state (s/keys :req-un [:flow/flows
:flow/threads-info
:printer/printers
:ui/selected-font-size-style-idx
:ui/selected-theme
:ui/selected-tool
:ui/extra-styles
:ui/jfx-nodes-index
:ui/jfx-stages
::pending-tasks-subscriptions
::connection-status
::local-mode?
::runtime-config
::debugger-config
::bookmarks
::data-windows]))
(defn initial-state [{:keys [theme styles local? port repl-type debugger-host ws-port runtime-host auto-update-ui? ui-timeout-millis call-tree-update?] :as config}]
{:flows {}
:printers {}
:selected-font-size-style-idx 0
:threads-info {}
:selected-theme (case theme
:light :light
:dark :dark
:auto ((requiring-resolve 'flow-storm.debugger.ui.utils/get-current-os-theme))
:light)
:selected-tool :tool-flows
:local-mode? (boolean local?)
:extra-styles styles
:jfx-nodes-index {}
:jfx-stages []
:pending-tasks-subscriptions {}
:runtime-config nil
:connection-status {:ws-ready? false
:repl-ready? false}
:debugger-config {:repl
(when port
(cond-> {:repl/kind :nrepl
:repl/type (or repl-type :clojure)
:repl/port port}
(#{:shadow} repl-type) (assoc :repl.cljs/build-id (:build-id config))))
:debugger-host (or debugger-host "localhost")
:debugger-ws-port (or ws-port 7722)
:runtime-host (or runtime-host "localhost")
:debug-mode? false
:auto-jump-on-exception? false
:ui-timeout-millis (or ui-timeout-millis 5000)
:auto-update-ui? (if-not (nil? auto-update-ui?) auto-update-ui? true)
:call-tree-update? (if-not (nil? call-tree-update?) call-tree-update? true)
:pprint-previews? false}
:bookmarks {}
:visualizers {}
:data-windows {}})
;; so linter doesn't complain
(declare state)
(declare fn-call-stats-map)
(declare flow-thread-indexers)
(defstate state
:start (fn [config] (atom (initial-state config)))
:stop (fn [] nil))
(defn local-mode? []
(:local-mode? @state))
(defn set-connection-status [conn-key status]
(let [k ({:ws :ws-ready?
:repl :repl-ready?} conn-key)]
(swap! state assoc-in [:connection-status k] status)))
(defn connection-status []
(get @state :connection-status))
(defn env-kind []
(let [state @state
k (get-in state [:runtime-config :env-kind])]
(if-not k
;; if we can determine if we are in :clj or :cljs because
;; we haven't connected to the runtime yet, lets guess from the debugger config
(let [{:keys [repl]} (:debugger-config state)]
(if (#{:shadow} (:repl/type repl))
:cljs
:clj))
k)))
(defn repl-config []
(get-in @state [:debugger-config :repl]))
(defn debugger-config []
(get @state :debugger-config))
(defn set-auto-jump-on-exception [enable?]
(swap! state assoc-in [:debugger-config :auto-jump-on-exception?] enable?))
(defn set-auto-update-ui [enable?]
(swap! state assoc-in [:debugger-config :auto-update-ui?] enable?))
(defn set-call-tree-update [enable?]
(swap! state assoc-in [:debugger-config :call-tree-update?] enable?))
(defn set-pprint-previews [enable?]
(swap! state assoc-in [:debugger-config :pprint-previews?] enable?))
(defn toggle-debug-mode []
(swap! state update-in [:debugger-config :debug-mode?] not))
(defn set-selected-tool [tool]
(swap! state assoc :selected-tool tool))
(defn selected-tool []
(get @state :selected-tool))
(defn set-runtime-config [config]
(swap! state assoc :runtime-config config))
(defn update-thread-info [thread-id info]
(swap! state assoc-in [:threads-info thread-id] info))
(defn get-threads-info []
(vals (get @state :threads-info)))
(defn get-thread-info [thread-id]
(get-in @state [:threads-info thread-id]))
(defn get-flow [flow-id]
(get-in @state [:flows flow-id]))
(defn create-thread [flow-id thread-id]
(swap! state assoc-in [:flows flow-id :flow/threads thread-id]
{:thread/id thread-id
:thread/curr-timeline-entry nil
:thread/callstack-tree-hidden-fns #{}
:thread/navigation-history {:head-pos 0
:history [{:fn-call-idx -1 ;; dummy entry
:idx -1}]}}))
(defn get-thread [flow-id thread-id]
(get-in @state [:flows flow-id :flow/threads thread-id]))
(defn remove-thread [flow-id thread-id]
(swap! state (fn [s]
(-> s
(update-in [:flows flow-id :flow/threads] dissoc thread-id)
(update :threads-info dissoc thread-id)))))
(defn create-flow [flow-id timestamp]
;; if a flow for `flow-id` already exist we discard it and
;; will be GCed
(swap! state assoc-in [:flows flow-id] {:flow/id flow-id
:flow/threads {}
:flow/exceptions {}
:flow/exceptions-limit-reached? false
:timestamp timestamp}))
(defn flow-threads-ids [flow-id]
(keys (get-in @state [:flows flow-id :flow/threads])))
(defn remove-flow [flow-id]
(doseq [tid (flow-threads-ids flow-id)]
(remove-thread flow-id tid))
(swap! state update :flows dissoc flow-id))
(defn all-flows-ids []
(keys (get @state :flows)))
(defn current-timeline-entry [flow-id thread-id]
(:thread/curr-timeline-entry (get-thread flow-id thread-id)))
(defn current-idx [flow-id thread-id]
(:idx (current-timeline-entry flow-id thread-id)))
(defn set-current-timeline-entry [flow-id thread-id entry]
(swap! state assoc-in [:flows flow-id :flow/threads thread-id :thread/curr-timeline-entry] entry))
(defn set-current-frame [flow-id thread-id frame-data]
(swap! state assoc-in [:flows flow-id :flow/threads thread-id :thread/curr-frame] frame-data))
(defn current-frame [flow-id thread-id]
(get-in @state [:flows flow-id :flow/threads thread-id :thread/curr-frame]))
(defn callstack-tree-hide-fn [flow-id thread-id fn-name fn-ns]
(swap! state update-in [:flows flow-id :flow/threads thread-id :thread/callstack-tree-hidden-fns] conj {:name fn-name :ns fn-ns}))
(defn callstack-tree-hidden? [flow-id thread-id fn-name fn-ns]
(let [hidden-set (get-in @state [:flows flow-id :flow/threads thread-id :thread/callstack-tree-hidden-fns])]
(contains? hidden-set {:name fn-name :ns fn-ns})))
(defn add-printer [flow-id form-id coord printer-data]
(swap! state assoc-in [:printers flow-id form-id coord] printer-data))
(defn printers [flow-id]
(get-in @state [:printers flow-id]))
(defn remove-printer [flow-id form-id coord]
(swap! state update-in [:printers flow-id form-id] dissoc coord))
(defn update-printer [flow-id form-id coord k new-val]
(swap! state assoc-in [:printers flow-id form-id coord k] new-val))
(def font-size-styles ["flowstorm/styles/font-size-sm.css"
"flowstorm/styles/font-size-md.css"
"flowstorm/styles/font-size-lg.css"
"flowstorm/styles/font-size-xl.css"])
(defn inc-font-size []
(-> (swap! state update :selected-font-size-style-idx
(fn [idx] (min (dec (count font-size-styles))
(inc idx))))
:selected-font-size-style-idx))
(defn dec-font-size []
(-> (swap! state update :selected-font-size-style-idx
(fn [idx] (max 0 (dec idx))))
:selected-font-size-style-idx))
(defn set-theme [theme]
(swap! state assoc :selected-theme theme))
(defn rotate-theme []
(swap! state update :selected-theme {:light :dark
:dark :light}))
(defn current-stylesheets []
(let [{:keys [selected-theme extra-styles selected-font-size-style-idx]} @state
plugins-styles (reduce (fn [styles {:keys [plugin/css-resource plugin/dark-css-resource plugin/light-css-resource]}]
(cond-> styles
css-resource
(conj (str (io/resource css-resource)))
(and (= selected-theme :dark) dark-css-resource)
(conj (str (io/resource dark-css-resource)))
(and (= selected-theme :light) light-css-resource)
(conj (str (io/resource light-css-resource)))))
[]
(plugins/plugins))
default-styles (str (io/resource "flowstorm/styles/styles.css"))
theme-base-styles (str (io/resource (case selected-theme
:dark "flowstorm/styles/theme_dark.css"
:light "flowstorm/styles/theme_light.css")))
font-size-style (-> (get font-size-styles selected-font-size-style-idx)
io/resource
str)
extra-styles (when extra-styles
(str (io/as-url (io/file extra-styles))))]
(cond-> [theme-base-styles
default-styles]
true (into plugins-styles)
true (conj font-size-style)
extra-styles (conj extra-styles))))
;;;;;;;;;;;;;;;;;;;;;;;
;; JFX objects index ;;
;;;;;;;;;;;;;;;;;;;;;;;
;; Because scene.lookup doesn't work if you lookup before a layout pass
;; So adding a node and looking it up quickly sometimes it doesn't work
;; ui objects references with a static application lifetime
;; objects stored here will not be collected ever
(defn store-obj
"Store an object on an index by [flow-id, thread-id, object-id].
Ment to be used to store javafx.scene.Node objects because
scene.lookup doesn't work if you lookup before a layout pass so
adding a node and looking it up quickly sometimes doesn't work.
More than one object can be stored under the same key.
Objects stored with `store-obj` can be retrived with `obj-lookup`."
([obj-id obj-ref]
(store-obj :no-flow nil obj-id obj-ref))
([flow-id obj-id obj-ref]
(store-obj flow-id nil obj-id obj-ref))
([flow-id thread-id obj-id obj-ref]
(let [k [flow-id thread-id obj-id]]
(swap! state update-in [:jfx-nodes-index k] conj obj-ref))))
(defn obj-lookup
"Retrieve objects stored by `store-obj`.
Returns a vector of all the objects stored under the requested key."
([obj-id]
(obj-lookup :no-flow nil obj-id))
([flow-id obj-id]
(obj-lookup flow-id nil obj-id))
([flow-id thread-id obj-id]
(let [k [flow-id thread-id obj-id]]
(get-in @state [:jfx-nodes-index k]))))
(defn clean-objs
"Clear objects stored by `store-obj` under a specific flow or
specific thread."
([] (clean-objs nil nil))
([flow-id]
(swap! state (fn [s]
(update s :jfx-nodes-index
(fn [objs]
(reduce-kv (fn [ret [fid tid oid] o]
(if (= fid flow-id)
ret
(assoc ret [fid tid oid] o)))
{}
objs))))))
([flow-id thread-id]
(swap! state (fn [s]
(update s :jfx-nodes-index
(fn [objs]
(reduce-kv (fn [ret [fid tid oid] o]
(if (and (= fid flow-id)
(= tid thread-id))
ret
(assoc ret [fid tid oid] o)))
{}
objs)))))))
;;;;;;;;;;;;;;;
;; Bookmarks ;;
;;;;;;;;;;;;;;;
(defn add-bookmark [{:keys [flow-id thread-id idx] :as bookmark}]
(swap! state assoc-in [:bookmarks [flow-id thread-id idx]] bookmark))
(defn remove-bookmark [flow-id thread-id idx]
(swap! state update :bookmarks dissoc [flow-id thread-id idx]))
(defn remove-bookmarks [flow-id]
(swap! state update :bookmarks
(fn [bookmarks]
(reduce-kv (fn [bks [fid :as bkey] btext]
(if (= fid flow-id)
bks
(assoc bks bkey btext)))
{}
bookmarks))))
(defn flow-bookmarks [flow-id]
(->> (:bookmarks @state)
vals
(filter (fn [bookmark]
(= flow-id (:flow-id bookmark))))))
(defn all-bookmarks []
(vals (:bookmarks @state)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Navigation undo system ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(def ^:dynamic *undo-redo-jump* false)
(defn update-nav-history
"Add to nav history if we are jumping to a different frame,
else update the head idx. If the head is not at the end, redo history
will be discarded."
[flow-id thread-id {:keys [fn-call-idx] :as tentry}]
(swap! state update-in [:flows flow-id :flow/threads thread-id :thread/navigation-history]
(fn [nav-hist]
(let [{:keys [history head-pos] :as nav-hist} (if (and (not *undo-redo-jump*)
(< (:head-pos nav-hist) (dec (count (:history nav-hist)))))
(-> nav-hist
(update :history subvec 0 (:head-pos nav-hist))
(update :head-pos dec))
nav-hist)
changing-frames? (not= fn-call-idx (get-in history [head-pos :fn-call-idx]))]
(if changing-frames?
(-> nav-hist
(update :history conj tentry)
(update :head-pos inc))
(assoc-in nav-hist [:history head-pos] tentry))))))
(defn current-nav-history-entry [flow-id thread-id]
(let [{:keys [history head-pos]} (get-in @state [:flows flow-id :flow/threads thread-id :thread/navigation-history])]
(get history head-pos)))
(defn undo-nav-history
"Move the nav history head back and return it's idx."
[flow-id thread-id]
(swap! state update-in [:flows flow-id :flow/threads thread-id :thread/navigation-history :head-pos]
(fn [p]
(if (> p 1)
(dec p)
p)))
(current-nav-history-entry flow-id thread-id))
(defn redo-nav-history
"Move the nav history head forward and return it's idx."
[flow-id thread-id]
(swap! state update-in [:flows flow-id :flow/threads thread-id :thread/navigation-history]
(fn [{:keys [history head-pos] :as h}]
(assoc h :head-pos (if (< (inc head-pos) (count history))
(inc head-pos)
head-pos))))
(current-nav-history-entry flow-id thread-id))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Function unwind (throws) ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn maybe-add-exception
"Track a limited amount of exceptions ex-data only if they doesn't already exist."
[{:keys [flow-id ex-hash] :as ex-data}]
(let [ex-ui-limit 100
exceptions (get-in @state [:flows flow-id :flow/exceptions])
exceptions-limit-reached? (get-in @state [:flows flow-id :flow/exceptions-limit-reached?])
ex-cnt (count exceptions)]
(cond
exceptions-limit-reached?
:ex-limit-passed
(= ex-cnt ex-ui-limit)
(do
(swap! state assoc-in [:flows flow-id :flow/exceptions-limit-reached?] true)
:ex-limit-reached)
(get exceptions ex-hash)
:ex-skipped
:else
(do
(swap! state assoc-in [:flows flow-id :flow/exceptions ex-hash] ex-data)
:ex-added))))
(defn flow-exceptions [flow-id]
(vals (get-in @state [:flows flow-id :flow/exceptions])))
;;;;;;;;;;;;;;;;
;; JFX Stages ;;
;;;;;;;;;;;;;;;;
(defn jfx-stages []
(get @state :jfx-stages))
(defn unregister-jfx-stage! [stg]
(swap! state update :jfx-stages (fn [stgs] (filterv #(not= % stg) stgs))))
(defn main-jfx-stage []
(-> @state :jfx-stages first))
(defn reset-theming []
(let [stages (jfx-stages)
new-stylesheets (current-stylesheets)]
(doseq [stage stages]
(let [scene (.getScene stage)
scene-stylesheets (.getStylesheets scene)]
(.clear scene-stylesheets)
(.addAll scene-stylesheets new-stylesheets)))))
(defn register-jfx-stage! [stg]
(swap! state update :jfx-stages conj stg)
(reset-theming))
;;;;;;;;;;;;;;;;;;;;
;; Functions list ;;
;;;;;;;;;;;;;;;;;;;;
(defn set-selected-function-list-fn [flow-id thread-id fn-call]
(swap! state assoc-in [:flows flow-id :flow/threads thread-id :thread.ui/selected-functions-list-fn] fn-call))
(defn get-selected-function-list-fn [flow-id thread-id]
(get-in @state [:flows flow-id :flow/threads thread-id :thread.ui/selected-functions-list-fn]))
;;;;;;;;;;;;;;;;;;
;; Data Windows ;;
;;;;;;;;;;;;;;;;;;
(defn data-window-create [dw-id nodes-map]
(swap! state assoc-in [:data-windows dw-id] (assoc nodes-map :stack ())))
(defn data-window [dw-id]
(get-in @state [:data-windows dw-id]))
(defn data-window-current-val [dw-id]
(-> @state :data-windows dw-id :stack first :val-data))
(defn data-window-remove [dw-id]
(swap! state update :data-windows dissoc dw-id))
(defn data-windows []
(:data-windows @state))
(defn data-window-push-frame [dw-id val-frame]
(swap! state update-in [:data-windows dw-id :stack] conj val-frame))
(defn data-window-update-top-frame
"Swaps the top frame of the dw-id data-window stack by new-frame-data.
Returns the replaced (old top) frame."
[dw-id new-frame-data]
(let [prev-top (peek (:stack (data-window dw-id)))]
(swap! state update-in [:data-windows dw-id :stack]
(fn [stack]
(conj (pop stack) (merge (peek stack) new-frame-data))))
prev-top))
(defn data-window-pop-stack-to-depth
"Pop the dw-id data-window stack so it is left with depth number of elements.
Returns popped elements."
[dw-id depth]
(let [stack (:stack (data-window dw-id))
pop-cnt (- (count stack) depth)
popped (take pop-cnt stack)]
(swap! state update-in [:data-windows dw-id :stack] pop-n pop-cnt)
popped))
;;;;;;;;;;;
;; Other ;;
;;;;;;;;;;;
(defn clojure-storm-env? []
(get-in @state [:runtime-config :storm?]))
(defn flow-storm-nrepl-middleware-available? []
(get-in @state [:runtime-config :flow-storm-nrepl-middleware?]))
================================================
FILE: src-dbg/flow_storm/debugger/tutorials/basics.clj
================================================
(ns flow-storm.debugger.tutorials.basics
(:require [flow-storm.debugger.ui.components :as ui]
[flow-storm.debugger.state :as dbg-state]
[flow-storm.debugger.ui.utils :as ui-utils]))
(def steps
[
"
Welcome to FlowStorm basics tutorial!
It will guide you over the basics and help you get started with FlowStorm.
If you started this repl using the command from the User's Guide QuickStart section, you find yourself in a standard (for the most) Clojure repl. You can run any expression and nothing should be different.
Nothing except single keyword evaluation.
Since evaluating single keywords on the repl is useless (they just return themselves) FlowStorm hijacks some of them as quick commands.
Try the :help command just to have a sense of what options are available, but don't focus too much on them now, we are going to cover them later.
Another command you already tried if you are here is the :dbg command, which starts the FlowStorm UI.
Any time you need the FlowStorm UI, evaluating :dbg is enough, no matter what namespace you are in, and you can always discard it by closing the window. It will take a couple of seconds the first time, but should open almost instantly from there on.
Now click on Next and I see you on the next slide!
"
"
Great! Since we are going to be using the UI throughout the tutorial, let's do it comfortably.
You can toggle the debugger's UI between light/dark themes by hitting (Ctrl-t) or using the View menu which also allows you to increase/decrease the font size. You can also toggle this tutorial theme by using the button at the bottom right corner, so go ahead and set yourself comfortable first.
Now we are ready! The first thing we are going to do is to create and jump to a namespace for our tutorial. So go ahead and paste the following code in your repl :
(ns tutorial)
Next thing we need is to tell FlowStorm what namespaces to instrument, so we can record their function's executions.
We can do this by setting some JVM properties before starting the repl or by using the FlowStorm Browser (second vertical tab from the top).
For most of your projects you probably want to setup instrumentation via JVM properties, so you don't need to tell FlowStorm what to instrument every time you start your project's repl, but this time we are going to use the browser.
Click on the Browser tab and look at the instrumentations bottom panel.
There are two ways of adding instrumentation prefixes from the browser. We can find our namespace on the namespaces browser, right click on them, and then select our prefix, or we can use the Add button in the instrumentations bottom panel.
Let's find our `tutorial` namespace on the list (you can filter them using the input at the top), right click on it, and click on `Add instr prefix for tutorial.*`
A window will popup asking if we want to reload all namespaces related to our new prefix, which we can cancel since we don't have anything in our ns yet.
Reloading is important when you add a prefix for namespaces with many functions already loaded. When you reload them you will be recompiling all the functions, and because there is a instrumentation prefix for them now, the functions will compile instrumented, so you can record their execution.
These are prefixes, so this means that for the `tutorial` prefix any code compiled under `tutorial`, `tutorial.server`, `tutorial.server.core`, etc, will get instrumented. Normally adding prefixes for the top namespace of your project and some libs you are interested in will be enough.
Now we are done with instrumentation setup for this tutorial, so let's go back to the Flows vertical tab , which is called the Flows tool and is what we are going to be using for the rest of the tutorial.
The next important control to learn about is the recording button, which is the second one on the Flows tool bar. Clicking it will toggle between recording/paused. Let's leave it on pause for now (you should leave it with the circle icon), we don't want anything to be recorded yet.
On the next slide we are going to start evaluating some code.
"
"
Great! Now we need some code to debug. Go ahead and evaluate the function below (you can copy and paste it on the repl) :
Now if you call the function, let's say (factorial 5) you should get 120 as your result, like in any normal repl.
But let's say you now want to understand this function execution. For this you just go to the UI and put the debugger in recording mode, then run (factorial 5) again.
This time you should see the debugger UI showing the code stepping tool, which we are going to cover next.
This tool is inside a `flow-0` tab, which we are going to ignore for now, and inside a thread tab, probably called `[1] main` if you are running your repl from a terminal.
This is the thread recordings exploration tab, which contains tools for exploring this thread execution.
On the next slide we will start exploring the execution.
Tip: On your daily work you can keep recording paused and turn it on right before executing something you are interested in.
"
"
The default tool is called the code stepping tool.
One thing to notice is that your factorial function code is showing there with some parts highlighted in pink, and also there is a bar at the top with some controls and some numbers.
The numbers show the position of the debugger in time for this specific thread. The number at the left is the current position, and the one on the right shows how many \"expressions executions\" were recorded so far for this thread. You can think of FlowStorm recording the activity on each thread as a timeline of execution steps, in which you can move around.
There are many ways of moving around in time in the `code stepping tool` but these are the basic ones :
By using the arrows on the second row of the controls panel. They are stepping controls similar to what you can find on most debuggers, but with the extra ability to also step backwards. Check out the tooltips to know how they move and give them a try.
By clicking on the highlights of the form. These are what FlowStorm captured as interesting debugging points for this frame. What it means by \"this frame\" is that clicking on a expression will take you to points in time inside the current function call. In the case of factorial that it is calling itself many times with clicks you get to navigate around the current call. You can click on any symbols and expressions. Whatever you click will get highlighted in green and the debugger will move to that point in time.
When navigating a particular call sometimes it's faster to click on the expression you want to see instead of clicking the next or prev step buttons many times.
Also notice that as you move through execution, two panels on the right change.
The top one shows a the current expression value while the bottom one shows all locals in scope.
There are two tabs for the top one, so two tools for displaying the current expression value. The first one is what FlowStorm calls a data window, which allows you to explore nested value and visualize them in different ways. The second just shows a pretty print of the value.
There is also a quick way to jump to the first execution of a function and it is by using the Quick jump box on the toolbar. It will auto complete with all the recorded functions and selecting one will take you there. It doesn't make much sense for this example since we have only one recorded function, but will be handy in more complex situations. Give it a try!
There are many more features on the code stepping tool but given this tutorial covers just the basics, we are going to skip them and jump right to another tool. The call tree tool. So go ahead and click on the second tab in the bottom left corner.
"
"
Welcome to the call tree tool.
This tool will show you a expandable tree of the functions calls, which will serve as an overview of your selected thread recordings. It will be very handy when trying to understand an end to end execution, helping you create a mental model of what is going on.
Expand the one that says `(factorial 5)` and keep expanding it. This already makes evident how this recursive factorial function works by calling itself. It shows you a tree of functions calls with its arguments.
You can also click on any node and the bottom two panels will show you a pretty print of the arguments vector on the left and of the return value on the right.
Now let's say you are interested in stepping through the code of your factorial function. We can travel just before `(factorial 2)` was called. For it, you will have to expand the nodes until you see the one that is calling the function with 2, and then double click it.
It should take you to the code stepping tool with the debugger positioned right at that point in time.
You can jump between this tools using the tabs at the bottom left corner. So clicking on the third one, we are now going to learn yet another tool. The functions list tool.
"
"
The functions list tool shows you all the functions next to how many times they have been called.
This is another way of looking at your recordings, which is very useful in some situations.
You should see at least a table with :
Functions
Calls
tutorial/factorial
6
This means that `tutorial/factorial` was called 6 times.
Selecting a function will show you a list of all recorded calls, together with their arguments and return values.
You can also use the checkboxes at the top to show/hide some of the arguments, which doesn't make sense on this case since we have a single argument, but can be handy in more noisy situations.
You can double click on any of the calls at the right to step over code at that specific point in time, or use the `args` or `ret` buttons to inspect the values if they have nested structure. We haven't look at the value inspector yet, but we will cover it soon.
Give it a shot, double click on the factorial call with arguments `[4]` to jump exactly to where `(factorial 4)` was called.
And that's it for the basic code exploring tools. There are more tools under the `More tools` menu but they are out of the scope of this tutorial!
Next we will learn about FlowStorm data exploring tools, so when you are ready click next.
"
"
Now we are going to learn about FlowStorm value exploring and visualization capabilities, but first let's clear all recordings and try a more interesting example.
For clearing our recordings you can go to the debugger window and hit `Ctlr-L` or you can also click on the trash can on the toolbar.
This is handy in two situations. First, to get rid of old recorded data to make everything cleaner, and second, to free the recorded data so the garbage collector can get rid of it.
Note: there is a bar at the bottom right corner that will show your max heap and how much of it is currently used. You can use this to keep an eye on your heap usage so you know when to clear or stop recording.
So go ahead, clear your recordings and then evaluate the next form :
(count (all-ns))
Now click on the highlighted expression of `(all-ns)` to see this expression value.
This Clojure function returns a list of all namespaces currently loaded, and as you can see on the top right panel, it is a sequence of namespaces objects.
This panel on the top right is called a data window. There are different places in FlowStorm that allows you to open a data window to explore your values.
From the top, we have the data window id, which we are going to skip for now (take look at the User's guide for a deeper look into data windows).
The next row shows the breadcrumbs, which will allow you to navigate back as you go deeper into nested values.
The following row contains the visualizer selector, the type of the value you are looking at, the def and finally the copy button.
Expanding the visualizers drowpdown will let you select all the currently defined visualizers for your current value.
Try clicking on some of the elements to navigate into your nested values. You can also navigate back by using the top bar breadcrumbs.
Now let's introduce a very powerful feature, the `def` button. You can take whatever value you are seeing in any FlowStorm panel back to your repl by giving it a name. You do this by clicking the `def` button, and it will ask you for a name. Let's say you named it `mydata`, now you can go to your repl and find it bound to the `user/mydata` var. You can define a value for the repl in any value panel you see in FlowStorm, not just in data windows.
There is also the `copy` value, which will let you put a pretty print of the current value on your clipboard.
"
"
For the last basic featur, let's see exceptions debugging.
First let's get rid of the recordings (Ctrl-L) and then eval these buggy function and call it :
You can always access the User's guide by clicking on the Help menu at the top.
Before finishing, keep in mind that even if FlowStorm is called a debugger (for lack of a better word) it was designed not just for chasing bugs, but for enhancing your interactive development experience by providing some visibility on what is going on as things execute, so it is pretty handy for other things like help you understanding a system from an execution POV or just running something and checking your assumptions.
Also here are some tips I've found for using FlowStorm efficiently :
Keep recording paused when not needed
Get rid of all the state (Ctrl-L) before executing the actions you are interested in recording
If you know the function name you want to see use the Quick jump box to quickly jump to it's first call.
If you see multiple calls, use the functions list to quickly move between them.
When exploring a system you know little about, the search, power-stepping and bookmarking tools can make wrapping your head around the new system much faster.
If there is any kind of looping, mapping or a function is called multiple times the printer tool is your friend.
Use the jvm options described in :help to configure it, so you don't record unnecessary stuff.
And that is all for the basics. If you find any issues or suggestions feel free to open a issue in https://github.com/flow-storm/flow-storm-debugger
This is the central repository for [FlowStorm](http://www.flow-storm.org/) an omniscient time travel debugger for Clojure and ClojureScript.
There are two ways of using it :
- [With ClojureStorm](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_clojurestorm) (recommended) : For dev, swap your Clojure compiler by ClojureStorm and get everything instrumented automatically
- [Vanilla FlowStorm](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_vanilla_flowstorm) : Just add FlowStorm to your dev classpath and instrument by re-evaluating forms
ClojureStorm is a fork of the official Clojure compiler that adds automatic instrumentation so you don't need to think about it (you can still disable it when you don't need it).
You use it by swapping the official Clojure compiler by ClojureStorm at dev time, using dev aliases or profiles.
# Artifacts
FlowStorm latest stable releases :
- The complete debugger (includes `flow-storm-inst`)
- `[com.github.flow-storm/flow-storm-dbg "4.5.9"]`
- A slimmer version with no GUI, to use it for Clojure or ClojureScript remote debugging
- `[com.github.flow-storm/flow-storm-inst "4.5.9"]`
ClojureStorm latest stable releases :
- Clojure 1.12
- `[com.github.flow-storm/clojure "1.12.4"]`
- Clojure 1.11
- `[com.github.flow-storm/clojure "1.11.4-10"]`
ClojureScriptStorm latest stable releases :
- ClojureScript 1.12.116
- `[com.github.flow-storm/clojurescript "1.12.134-3"]`
- ClojureScript 1.11.132
- `[com.github.flow-storm/clojurescript "1.11.132-9"]`
# Prerequisites
- jdk17+ (if you still need to run it with jdk11 take a look at [here](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_run_with_jdk_11))
- clojure 1.11.0+
- clojure 1.10.* only supported if you use it from source, like `{:git/url "https://github.com/flow-storm/flow-storm-debugger" :git/sha "..."}`
# QuickStart and Documentation
If you want to use it with Clojure checkout the [Clojure QuickStart guide](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_clojure)
or the [ClojureScript QuickStart Guide](https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_clojurescript) if you are using ClojureScript.
Please refer to the [user's guide](https://flow-storm.github.io/flow-storm-debugger/user_guide.html) for a list of features and how to use them.
# ClojureStorm and ClojureScriptStorm
*ClojureStorm* is a dev compiler. It is a fork of the official Clojure compiler enhanced with automatic instrumentation.
To use it, you just swap it with your normal Clojure compiler at dev time (by using deps cli aliases or lein profiles) to improve
your development experience, while making sure you use your normal compiler for everything else (tests and production).
*ClojureScriptStorm* is the same as ClojureStorm but for ClojureScript, so a fork of the official ClojureScript compiler is enhanced with automatic instrumentation.
ClojureStorm sources are here : https://github.com/flow-storm/clojure
ClojureScriptStorm sources are here : https://github.com/flow-storm/clojurescript
# Features
Flow storm debugger is packed with a ton of features, which the [user's guide](https://flow-storm.github.io/flow-storm-debugger/user_guide.html)
covers in detail.
## Information for developers
If you want to enhance, fix, debug, or just learn about the internals of FlowStorm take a look at
[here](./docs/dev_notes.md)
## Some demo videos (newers at the top)
- [FlowStorm demo at Clojure Apropos](https://www.youtube.com/watch?v=a-PrBjlBdw8)
- [ClojureScript compiler fun with FlowStorm](https://www.youtube.com/watch?v=YYHRx3EnPmg)
- [Don't fear the storm](https://www.youtube.com/watch?v=CspQX_R0NbM)
- [Clojure visual-tools 29 - FlowStorm 4.1 workflows](https://www.youtube.com/watch?v=9nY25hwzWRc)
- [Clojure web apps with FlowStorm 3.17](https://www.youtube.com/watch?v=h8AFpZkAwPo)
- [Reifying execution, the interactive programming missing piece](https://www.youtube.com/watch?v=BuSpMvVU7j4&t=1394s)
- [FlowStorm printer demo](https://www.youtube.com/watch?v=06-MA4HSS24)
- [Smashing a real ClojureScript bug with FlowStorm](https://www.youtube.com/watch?v=4VXT-RHHuvI)
- [Debugging Clojure with FlowStorm 3.6](https://www.youtube.com/watch?v=Mmr1nO6uMzc)
- [Searching and following values](https://www.youtube.com/watch?v=CwXhy-QsZHw)
- [Show me your REPL episode](https://www.youtube.com/watch?v=2nH59edD5Uo)
- [Debugging Clojure with FlowStorm](https://www.youtube.com/watch?v=PbGVTVs1yiU)
- [Debugging ClojureScript with FlowStorm](https://www.youtube.com/watch?v=jMYl32lnMhI)
- [Presentation at London Clojurians](https://www.youtube.com/watch?v=A3AzlqNwUXc)
- [Flows basics](https://www.youtube.com/watch?v=YnpQMrkj4v8)
- [Instrumenting libraries](https://youtu.be/YnpQMrkj4v8?t=332)
- [Debugging the ClojureScript compiler](https://youtu.be/YnpQMrkj4v8?t=533)
- [Browser](https://www.youtube.com/watch?v=cnLwRzxrKDk)
- [Def button](https://youtu.be/cnLwRzxrKDk?t=103)
## FAQ
### Clojure has the repl, why does it need a debugger?
In [this talk](https://www.youtube.com/watch?v=A3AzlqNwUXc&t=934s) I tried to argue that even as amazing as it is to have a repl to poke around,
there are some inconveniences that I think can be greatly improved by a debugger.
- Defining function parameters and locals with def (for sub form execution) isn't easy for complex values
- When functions contains loops, maps, filters, etc with anonymous functions is hard to capture every value for further inspection.
- Being Clojure a dynamic lang, running the program in my head isn't easy if I'm not familiar with the code base
- Adding prints (or tap>) is inconvenient because you need to guess where the problem probably is first.
- Debugging statements needs to be constantly typed and removed which gets repetitive and annoying
- Exploring complex values at the console is tedious, that is why tools like portal, reveal, rebl, etc exist.
Some of the issues there can be alleviated by adding libraries like scope capture, and portal but it isn't straight forward to integrate them
and even if it was IMHO there is a lot to be gained from a proper integrated debugging system.
So I want to stop guessing and a tool that allows me to see what is happening when a Clojure program runs, be it a small expression or an entire codebase.
I want it to be useful for cases when I'm chasing a bug or when I just want to understand how something works.
I also think some Clojure constraints, like immutability and being expression based, allows us to go beyond repl poking and steppers.
We can record everything that happens when a program runs and then inspect the execution using multiple tools (being a stepper one of them) and fully
integrating it with the repl which also enhance the repl poking experience.
### What's that magic? How does it work?
The idea behind FlowStorm is pretty simple :
- Instrument Clojure code
- Run it
- Record everything that happens on each thread during execution into timelines
- Provide a GUI with multiple tools to explore the recorded values and execution flows
The interesting part here I guess are instrumentation and recording.
FlowStorm can instrument your code in two ways :
- The ClojureStorm way (recommended for Clojure) which swaps your official Clojure compiler with a patched one (only for dev) that emits extra JVM bytecode everytime you compile something
to record everything that is happening. This method provides automatic instrumentation everywhere, which is very practical. You still get to un-instrument things if you need to do
things like measure performance, which isn't going to be accurate with the extra bytecode.
- The vanilla way (can be used for Clojure and is the only option for ClojureScript), that just grabs the Clojure source expressions you are interested in, walks the AST, instruments it, and re-evals the
instrumented version through the repl. You do this by using reader tags (like `#trace (defn foo [...] ...)`) or by using the FlowStorm browser tab to instrument entire namespaces.
Doesn't matter which method instrumented the code, when it runs it will record every expression in a timeline.
Because Clojure is expression based and most data is immutable, recording is just retaining JVM references together with the source coordinate of each expression. This is pretty fast and a simple way of recording execution.
The timeline is a structure that provides efficient insertion, and efficient access sequentially and as a functions call tree.
You can see some diagrams here : https://flow-storm.github.io/flow-storm-debugger/user_guide.html#_internals_diagrams_and_documentation
### Isn't recording everything too expensive? Like, does it work with something like a game loop?
The answer here is it depends. First, not everything needs to be recorded all the time. FlowStorm provides easy controls for start/stop recording, instrumenting/un-instrumenting stuff,
freeing recordings, and for keeping an eye on the heap space which you can also control via JVM parameters.
For a lot of applications it is probably fine even if you keep recording everything. For applications that generate a lot of recordings you can keep the recording off and just enable it
with one click right before executing the action you want to record, and then turn it off again.
For things like game loops you can also use the thread breakpoints functionality which provides a way of pausing/resuming threads at specific points so you have more control on how they
execute. Combining that with start/stopping recording is probably enough to debug this kind of application.
### Aren't omniscient debuggers slow?
Omniscient debuggers have a reputation of being slow, and I think a bunch of them are. IMHO a lot of the slowness comes from how things need to be recorded and replayed on mutable languages.
For Clojure, just retaining references is enough, and looking into a value at any point in time is just looking at it. Since most references point to immutable values, they don't need to be
reconstructed by applying state changes for each step.
There are mutable language omniscient debuggers that implement stepping backwards fast enough to be useful, but trying to implement things like searching or following values is probably going to be
much slower since everything needs to be reconstructed at each step.
### How does it compare to other Clojure debuggers?
There are many comparison points with different Clojure debugging tools, I'll try to list some of them here.
If you feel something here is unfair or needs to be clarified please open an issue or submit a PR. Also if you would like to see here how it compares with any other debugging tool let me know.
The first big difference is that FlowStorm works for Clojure and ClojureScript while most debuggers like Cider/VSCode/Cursive only work for Clojure. So once you learn
to use FlowStorm you can use it with both languages.
The second imho big difference is that most debugging tools are designed to understand small pieces of execution you could be interested in, while FlowStorm
is designed to look at a program execution as a whole, to be used not only while chasing bugs but also as a development companion to help you reason about whatever
system you are running.
Most tools like Cider, Cursive or VScode debuggers are blocking steppers and require you to add breakpoints to specific places, which is okay if you have an idea
already of where the bug could be located, but they fall short if you aren't sure, or you just want to use the tool to understand an entire codebase execution.
#### FlowStorm VS Cursive (or Browser's for ClojureScript) debuggers
I think the main difference here is FlowStorm being expression and value oriented while the Cursive and Browser's one being line and memory poking oriented,
which I don't think is very useful in Clojure[Script]. If you want to experience what I mean, try to debug this exception using Cursive vs FlowStorm/Cider/VSCode :
```
(defn foo [n]
(->> (range n)
(filter odd?)
(partition-all 2)
(map second)
(drop 10)
(reduce +)))
(foo 70)
```
On the other side, the nice thing about the Cursive debugger is that you can use it to step over Java code which is useful if you have a mixed lang codebase. But for those cases
you can always use both.
#### FlowStorm VS Cider/VSCode debuggers
Cider and VSCode debuggers are steppers, so I'll only be comparing it against the stepping and value inspection capabilities of FlowStorm, but keep in mind that FlowStorm provides
many more features than stepping. Cider provide some tracing capabilities to trace entire namespaces but it relies on printing to the console which I haven't found useful outside of very specific situations, so
I don't think it is useful to understand an entire codebase.
For the steppers, the main difference is that Cider/VSCode are blocking debuggers, which are nice in some situations but only provide stepping forward, while FlowStorm allows you to
step in many different ways.
Another difference is that currently on Cider/VSCode you can't step into functions unless you have previously instrumented them, which for the most isn't a problem in FlowStorm if you are using ClojureStorm.
A nice thing about Cider/VSCode steppers is that you get to step over your editor source code, which can also be done in FlowStorm if you use an integration like CiderStorm.
There is also some code that currently can't be stepped with Cider/VSCode that can be with FlowStorm. This is code that happens to be inside literal sets of maps with more than 8 keys.
If you want to compare it try stepping code like this in both debuggers :
```
(defn bla []
{(+ 1 2) (first #{3})
2 (+ 1 1)
4 (+ 2 2)
6 (+ 3 3)
8 (+ 4 4)
10 (+ 5 5)
12 (+ 6 6)
14 (+ 7 7)
16 (+ 8 8)})
(bla)
```
## What to do when things don't work?
Please create a [issue](https://github.com/flow-storm/flow-storm-debugger/issues) if you think you found a bug.
If you are not sure you can ask in :
- [#flow-storm slack channel](https://clojurians.slack.com/archives/C03KZ3XT0CF)
- [github discussions](https://github.com/flow-storm/flow-storm-debugger/discussions)
## Acknowledgements
Big thanks to [Roam Research](https://roamresearch.com/), [Nubank](https://nubank.com.br/) and all previous and current sponsors.
Thanks to [Cider](https://github.com/clojure-emacs/cider/) debugger for inspiration and some clever ideas for code instrumentation.
================================================
FILE: UNLICENSE
================================================
This is free and unencumbered software released into the public domain.
Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.
In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
For more information, please refer to 
