Showing preview only (340K chars total). Download the full file or copy to clipboard to get everything.
Repository: upa/mscp
Branch: main
Commit: 1313853d7d30
Files: 71
Total size: 320.7 KB
Directory structure:
gitextract_bck5e2hc/
├── .clang-format
├── .dockerignore
├── .github/
│ └── workflows/
│ ├── build-freebsd.yml
│ ├── build-macos.yml
│ ├── build-ubuntu.yml
│ ├── codeql.yml
│ ├── release.yml
│ └── test.yml
├── .gitignore
├── .gitmodules
├── CMakeLists.txt
├── Dockerfile/
│ ├── README.md
│ ├── almalinux-9.3.Dockerfile
│ ├── alpine-3.22.Dockerfile
│ ├── arch-base.Dockerfile
│ ├── build-deb.Dockerfile
│ ├── build-srpm.Dockerfile
│ ├── rocky-8.9.Dockerfile
│ ├── rocky-9.3.Dockerfile
│ ├── ubuntu-20.04.Dockerfile
│ ├── ubuntu-22.04.Dockerfile
│ └── ubuntu-24.04.Dockerfile
├── Doxyfile
├── LICENSE
├── README.md
├── VERSION
├── conanfile.txt
├── doc/
│ ├── RELEASE.md
│ ├── mscp.1.in
│ └── mscp.rst
├── include/
│ ├── config.h.in
│ └── mscp.h
├── patch/
│ ├── README.md
│ ├── libssh-0.10.4.patch
│ ├── libssh-0.10.6-2-g6f1b1e76.patch
│ ├── libssh-0.10.6.patch
│ ├── libssh-0.11.2.patch
│ └── libssh-0.9.6.patch
├── rpm/
│ ├── .gitignore
│ └── mscp.spec.in
├── scripts/
│ ├── install-build-deps.sh
│ └── test-in-container.sh
├── src/
│ ├── atomic.h
│ ├── bwlimit.c
│ ├── bwlimit.h
│ ├── checkpoint.c
│ ├── checkpoint.h
│ ├── fileops.c
│ ├── fileops.h
│ ├── main.c
│ ├── minmax.h
│ ├── mscp.c
│ ├── openbsd-compat/
│ │ ├── openbsd-compat.h
│ │ └── strlcat.c
│ ├── path.c
│ ├── path.h
│ ├── platform.c
│ ├── platform.h
│ ├── pool.c
│ ├── pool.h
│ ├── print.c
│ ├── print.h
│ ├── ssh.c
│ ├── ssh.h
│ ├── strerrno.c
│ └── strerrno.h
└── test/
├── .gitignore
├── README.md
├── conftest.py
├── test_e2e.py
└── util.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .clang-format
================================================
# SPDX-License-Identifier: GPL-2.0
#
# clang-format configuration file. Intended for clang-format >= 11.
#
# For more information, see:
#
# Documentation/process/clang-format.rst
# https://clang.llvm.org/docs/ClangFormat.html
# https://clang.llvm.org/docs/ClangFormatStyleOptions.html
#
# clang-format configuration for Linux kernel, except that ColumnLimit is 90
---
AccessModifierOffset: -4
AlignAfterOpenBracket: Align
AlignConsecutiveAssignments: false
AlignConsecutiveDeclarations: false
AlignEscapedNewlines: Left
AlignOperands: true
AlignTrailingComments: false
AllowAllParametersOfDeclarationOnNextLine: false
AllowShortBlocksOnASingleLine: false
AllowShortCaseLabelsOnASingleLine: false
AllowShortFunctionsOnASingleLine: None
AllowShortIfStatementsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AlwaysBreakAfterDefinitionReturnType: None
AlwaysBreakAfterReturnType: None
AlwaysBreakBeforeMultilineStrings: false
AlwaysBreakTemplateDeclarations: false
BinPackArguments: true
BinPackParameters: true
BraceWrapping:
AfterClass: false
AfterControlStatement: false
AfterEnum: false
AfterFunction: true
AfterNamespace: true
AfterObjCDeclaration: false
AfterStruct: false
AfterUnion: false
AfterExternBlock: false
BeforeCatch: false
BeforeElse: false
IndentBraces: false
SplitEmptyFunction: true
SplitEmptyRecord: true
SplitEmptyNamespace: true
BreakBeforeBinaryOperators: None
BreakBeforeBraces: Custom
BreakBeforeInheritanceComma: false
BreakBeforeTernaryOperators: false
BreakConstructorInitializersBeforeComma: false
BreakConstructorInitializers: BeforeComma
BreakAfterJavaFieldAnnotations: false
BreakStringLiterals: false
ColumnLimit: 90
CommentPragmas: '^ IWYU pragma:'
CompactNamespaces: false
ConstructorInitializerAllOnOneLineOrOnePerLine: false
ConstructorInitializerIndentWidth: 8
ContinuationIndentWidth: 8
Cpp11BracedListStyle: false
DerivePointerAlignment: false
DisableFormat: false
ExperimentalAutoDetectBinPacking: false
FixNamespaceComments: false
# Taken from:
# git grep -h '^#define [^[:space:]]*for_each[^[:space:]]*(' include/ tools/ \
# | sed "s,^#define \([^[:space:]]*for_each[^[:space:]]*\)(.*$, - '\1'," \
# | LC_ALL=C sort -u
ForEachMacros:
- '__ata_qc_for_each'
- '__bio_for_each_bvec'
- '__bio_for_each_segment'
- '__evlist__for_each_entry'
- '__evlist__for_each_entry_continue'
- '__evlist__for_each_entry_from'
- '__evlist__for_each_entry_reverse'
- '__evlist__for_each_entry_safe'
- '__for_each_mem_range'
- '__for_each_mem_range_rev'
- '__for_each_thread'
- '__hlist_for_each_rcu'
- '__map__for_each_symbol_by_name'
- '__pci_bus_for_each_res0'
- '__pci_bus_for_each_res1'
- '__pci_dev_for_each_res0'
- '__pci_dev_for_each_res1'
- '__perf_evlist__for_each_entry'
- '__perf_evlist__for_each_entry_reverse'
- '__perf_evlist__for_each_entry_safe'
- '__rq_for_each_bio'
- '__shost_for_each_device'
- '__sym_for_each'
- 'apei_estatus_for_each_section'
- 'ata_for_each_dev'
- 'ata_for_each_link'
- 'ata_qc_for_each'
- 'ata_qc_for_each_raw'
- 'ata_qc_for_each_with_internal'
- 'ax25_for_each'
- 'ax25_uid_for_each'
- 'bio_for_each_bvec'
- 'bio_for_each_bvec_all'
- 'bio_for_each_folio_all'
- 'bio_for_each_integrity_vec'
- 'bio_for_each_segment'
- 'bio_for_each_segment_all'
- 'bio_list_for_each'
- 'bip_for_each_vec'
- 'bond_for_each_slave'
- 'bond_for_each_slave_rcu'
- 'bpf_for_each'
- 'bpf_for_each_reg_in_vstate'
- 'bpf_for_each_reg_in_vstate_mask'
- 'bpf_for_each_spilled_reg'
- 'bpf_object__for_each_map'
- 'bpf_object__for_each_program'
- 'btree_for_each_safe128'
- 'btree_for_each_safe32'
- 'btree_for_each_safe64'
- 'btree_for_each_safel'
- 'card_for_each_dev'
- 'cgroup_taskset_for_each'
- 'cgroup_taskset_for_each_leader'
- 'cpu_aggr_map__for_each_idx'
- 'cpufreq_for_each_efficient_entry_idx'
- 'cpufreq_for_each_entry'
- 'cpufreq_for_each_entry_idx'
- 'cpufreq_for_each_valid_entry'
- 'cpufreq_for_each_valid_entry_idx'
- 'css_for_each_child'
- 'css_for_each_descendant_post'
- 'css_for_each_descendant_pre'
- 'damon_for_each_region'
- 'damon_for_each_region_from'
- 'damon_for_each_region_safe'
- 'damon_for_each_scheme'
- 'damon_for_each_scheme_safe'
- 'damon_for_each_target'
- 'damon_for_each_target_safe'
- 'damos_for_each_filter'
- 'damos_for_each_filter_safe'
- 'data__for_each_file'
- 'data__for_each_file_new'
- 'data__for_each_file_start'
- 'device_for_each_child_node'
- 'displayid_iter_for_each'
- 'dma_fence_array_for_each'
- 'dma_fence_chain_for_each'
- 'dma_fence_unwrap_for_each'
- 'dma_resv_for_each_fence'
- 'dma_resv_for_each_fence_unlocked'
- 'do_for_each_ftrace_op'
- 'drm_atomic_crtc_for_each_plane'
- 'drm_atomic_crtc_state_for_each_plane'
- 'drm_atomic_crtc_state_for_each_plane_state'
- 'drm_atomic_for_each_plane_damage'
- 'drm_client_for_each_connector_iter'
- 'drm_client_for_each_modeset'
- 'drm_connector_for_each_possible_encoder'
- 'drm_exec_for_each_locked_object'
- 'drm_exec_for_each_locked_object_reverse'
- 'drm_for_each_bridge_in_chain'
- 'drm_for_each_connector_iter'
- 'drm_for_each_crtc'
- 'drm_for_each_crtc_reverse'
- 'drm_for_each_encoder'
- 'drm_for_each_encoder_mask'
- 'drm_for_each_fb'
- 'drm_for_each_legacy_plane'
- 'drm_for_each_plane'
- 'drm_for_each_plane_mask'
- 'drm_for_each_privobj'
- 'drm_gem_for_each_gpuva'
- 'drm_gem_for_each_gpuva_safe'
- 'drm_gpuva_for_each_op'
- 'drm_gpuva_for_each_op_from_reverse'
- 'drm_gpuva_for_each_op_safe'
- 'drm_gpuvm_for_each_va'
- 'drm_gpuvm_for_each_va_range'
- 'drm_gpuvm_for_each_va_range_safe'
- 'drm_gpuvm_for_each_va_safe'
- 'drm_mm_for_each_hole'
- 'drm_mm_for_each_node'
- 'drm_mm_for_each_node_in_range'
- 'drm_mm_for_each_node_safe'
- 'dsa_switch_for_each_available_port'
- 'dsa_switch_for_each_cpu_port'
- 'dsa_switch_for_each_cpu_port_continue_reverse'
- 'dsa_switch_for_each_port'
- 'dsa_switch_for_each_port_continue_reverse'
- 'dsa_switch_for_each_port_safe'
- 'dsa_switch_for_each_user_port'
- 'dsa_tree_for_each_cpu_port'
- 'dsa_tree_for_each_user_port'
- 'dsa_tree_for_each_user_port_continue_reverse'
- 'dso__for_each_symbol'
- 'dsos__for_each_with_build_id'
- 'elf_hash_for_each_possible'
- 'elf_symtab__for_each_symbol'
- 'evlist__for_each_cpu'
- 'evlist__for_each_entry'
- 'evlist__for_each_entry_continue'
- 'evlist__for_each_entry_from'
- 'evlist__for_each_entry_reverse'
- 'evlist__for_each_entry_safe'
- 'flow_action_for_each'
- 'for_each_acpi_consumer_dev'
- 'for_each_acpi_dev_match'
- 'for_each_active_dev_scope'
- 'for_each_active_drhd_unit'
- 'for_each_active_iommu'
- 'for_each_active_route'
- 'for_each_aggr_pgid'
- 'for_each_and_bit'
- 'for_each_andnot_bit'
- 'for_each_available_child_of_node'
- 'for_each_bench'
- 'for_each_bio'
- 'for_each_board_func_rsrc'
- 'for_each_btf_ext_rec'
- 'for_each_btf_ext_sec'
- 'for_each_bvec'
- 'for_each_card_auxs'
- 'for_each_card_auxs_safe'
- 'for_each_card_components'
- 'for_each_card_dapms'
- 'for_each_card_pre_auxs'
- 'for_each_card_prelinks'
- 'for_each_card_rtds'
- 'for_each_card_rtds_safe'
- 'for_each_card_widgets'
- 'for_each_card_widgets_safe'
- 'for_each_cgroup_storage_type'
- 'for_each_child_of_node'
- 'for_each_clear_bit'
- 'for_each_clear_bit_from'
- 'for_each_clear_bitrange'
- 'for_each_clear_bitrange_from'
- 'for_each_cmd'
- 'for_each_cmsghdr'
- 'for_each_collection'
- 'for_each_comp_order'
- 'for_each_compatible_node'
- 'for_each_component_dais'
- 'for_each_component_dais_safe'
- 'for_each_conduit'
- 'for_each_console'
- 'for_each_console_srcu'
- 'for_each_cpu'
- 'for_each_cpu_and'
- 'for_each_cpu_andnot'
- 'for_each_cpu_or'
- 'for_each_cpu_wrap'
- 'for_each_dapm_widgets'
- 'for_each_dedup_cand'
- 'for_each_dev_addr'
- 'for_each_dev_scope'
- 'for_each_dma_cap_mask'
- 'for_each_dpcm_be'
- 'for_each_dpcm_be_rollback'
- 'for_each_dpcm_be_safe'
- 'for_each_dpcm_fe'
- 'for_each_drhd_unit'
- 'for_each_dss_dev'
- 'for_each_efi_memory_desc'
- 'for_each_efi_memory_desc_in_map'
- 'for_each_element'
- 'for_each_element_extid'
- 'for_each_element_id'
- 'for_each_endpoint_of_node'
- 'for_each_event'
- 'for_each_event_tps'
- 'for_each_evictable_lru'
- 'for_each_fib6_node_rt_rcu'
- 'for_each_fib6_walker_rt'
- 'for_each_free_mem_pfn_range_in_zone'
- 'for_each_free_mem_pfn_range_in_zone_from'
- 'for_each_free_mem_range'
- 'for_each_free_mem_range_reverse'
- 'for_each_func_rsrc'
- 'for_each_gpiochip_node'
- 'for_each_group_evsel'
- 'for_each_group_evsel_head'
- 'for_each_group_member'
- 'for_each_group_member_head'
- 'for_each_hstate'
- 'for_each_if'
- 'for_each_inject_fn'
- 'for_each_insn'
- 'for_each_insn_prefix'
- 'for_each_intid'
- 'for_each_iommu'
- 'for_each_ip_tunnel_rcu'
- 'for_each_irq_nr'
- 'for_each_lang'
- 'for_each_link_codecs'
- 'for_each_link_cpus'
- 'for_each_link_platforms'
- 'for_each_lru'
- 'for_each_matching_node'
- 'for_each_matching_node_and_match'
- 'for_each_media_entity_data_link'
- 'for_each_mem_pfn_range'
- 'for_each_mem_range'
- 'for_each_mem_range_rev'
- 'for_each_mem_region'
- 'for_each_member'
- 'for_each_memory'
- 'for_each_migratetype_order'
- 'for_each_missing_reg'
- 'for_each_mle_subelement'
- 'for_each_mod_mem_type'
- 'for_each_net'
- 'for_each_net_continue_reverse'
- 'for_each_net_rcu'
- 'for_each_netdev'
- 'for_each_netdev_continue'
- 'for_each_netdev_continue_rcu'
- 'for_each_netdev_continue_reverse'
- 'for_each_netdev_dump'
- 'for_each_netdev_feature'
- 'for_each_netdev_in_bond_rcu'
- 'for_each_netdev_rcu'
- 'for_each_netdev_reverse'
- 'for_each_netdev_safe'
- 'for_each_new_connector_in_state'
- 'for_each_new_crtc_in_state'
- 'for_each_new_mst_mgr_in_state'
- 'for_each_new_plane_in_state'
- 'for_each_new_plane_in_state_reverse'
- 'for_each_new_private_obj_in_state'
- 'for_each_new_reg'
- 'for_each_node'
- 'for_each_node_by_name'
- 'for_each_node_by_type'
- 'for_each_node_mask'
- 'for_each_node_state'
- 'for_each_node_with_cpus'
- 'for_each_node_with_property'
- 'for_each_nonreserved_multicast_dest_pgid'
- 'for_each_numa_hop_mask'
- 'for_each_of_allnodes'
- 'for_each_of_allnodes_from'
- 'for_each_of_cpu_node'
- 'for_each_of_pci_range'
- 'for_each_old_connector_in_state'
- 'for_each_old_crtc_in_state'
- 'for_each_old_mst_mgr_in_state'
- 'for_each_old_plane_in_state'
- 'for_each_old_private_obj_in_state'
- 'for_each_oldnew_connector_in_state'
- 'for_each_oldnew_crtc_in_state'
- 'for_each_oldnew_mst_mgr_in_state'
- 'for_each_oldnew_plane_in_state'
- 'for_each_oldnew_plane_in_state_reverse'
- 'for_each_oldnew_private_obj_in_state'
- 'for_each_online_cpu'
- 'for_each_online_node'
- 'for_each_online_pgdat'
- 'for_each_or_bit'
- 'for_each_path'
- 'for_each_pci_bridge'
- 'for_each_pci_dev'
- 'for_each_pcm_streams'
- 'for_each_physmem_range'
- 'for_each_populated_zone'
- 'for_each_possible_cpu'
- 'for_each_present_blessed_reg'
- 'for_each_present_cpu'
- 'for_each_prime_number'
- 'for_each_prime_number_from'
- 'for_each_probe_cache_entry'
- 'for_each_process'
- 'for_each_process_thread'
- 'for_each_prop_codec_conf'
- 'for_each_prop_dai_codec'
- 'for_each_prop_dai_cpu'
- 'for_each_prop_dlc_codecs'
- 'for_each_prop_dlc_cpus'
- 'for_each_prop_dlc_platforms'
- 'for_each_property_of_node'
- 'for_each_reg'
- 'for_each_reg_filtered'
- 'for_each_reloc'
- 'for_each_reloc_from'
- 'for_each_requested_gpio'
- 'for_each_requested_gpio_in_range'
- 'for_each_reserved_mem_range'
- 'for_each_reserved_mem_region'
- 'for_each_rtd_codec_dais'
- 'for_each_rtd_components'
- 'for_each_rtd_cpu_dais'
- 'for_each_rtd_dais'
- 'for_each_sband_iftype_data'
- 'for_each_script'
- 'for_each_sec'
- 'for_each_set_bit'
- 'for_each_set_bit_from'
- 'for_each_set_bit_wrap'
- 'for_each_set_bitrange'
- 'for_each_set_bitrange_from'
- 'for_each_set_clump8'
- 'for_each_sg'
- 'for_each_sg_dma_page'
- 'for_each_sg_page'
- 'for_each_sgtable_dma_page'
- 'for_each_sgtable_dma_sg'
- 'for_each_sgtable_page'
- 'for_each_sgtable_sg'
- 'for_each_sibling_event'
- 'for_each_sta_active_link'
- 'for_each_subelement'
- 'for_each_subelement_extid'
- 'for_each_subelement_id'
- 'for_each_sublist'
- 'for_each_subsystem'
- 'for_each_supported_activate_fn'
- 'for_each_supported_inject_fn'
- 'for_each_sym'
- 'for_each_test'
- 'for_each_thread'
- 'for_each_token'
- 'for_each_unicast_dest_pgid'
- 'for_each_valid_link'
- 'for_each_vif_active_link'
- 'for_each_vma'
- 'for_each_vma_range'
- 'for_each_vsi'
- 'for_each_wakeup_source'
- 'for_each_zone'
- 'for_each_zone_zonelist'
- 'for_each_zone_zonelist_nodemask'
- 'func_for_each_insn'
- 'fwnode_for_each_available_child_node'
- 'fwnode_for_each_child_node'
- 'fwnode_for_each_parent_node'
- 'fwnode_graph_for_each_endpoint'
- 'gadget_for_each_ep'
- 'genradix_for_each'
- 'genradix_for_each_from'
- 'genradix_for_each_reverse'
- 'hash_for_each'
- 'hash_for_each_possible'
- 'hash_for_each_possible_rcu'
- 'hash_for_each_possible_rcu_notrace'
- 'hash_for_each_possible_safe'
- 'hash_for_each_rcu'
- 'hash_for_each_safe'
- 'hashmap__for_each_entry'
- 'hashmap__for_each_entry_safe'
- 'hashmap__for_each_key_entry'
- 'hashmap__for_each_key_entry_safe'
- 'hctx_for_each_ctx'
- 'hists__for_each_format'
- 'hists__for_each_sort_list'
- 'hlist_bl_for_each_entry'
- 'hlist_bl_for_each_entry_rcu'
- 'hlist_bl_for_each_entry_safe'
- 'hlist_for_each'
- 'hlist_for_each_entry'
- 'hlist_for_each_entry_continue'
- 'hlist_for_each_entry_continue_rcu'
- 'hlist_for_each_entry_continue_rcu_bh'
- 'hlist_for_each_entry_from'
- 'hlist_for_each_entry_from_rcu'
- 'hlist_for_each_entry_rcu'
- 'hlist_for_each_entry_rcu_bh'
- 'hlist_for_each_entry_rcu_notrace'
- 'hlist_for_each_entry_safe'
- 'hlist_for_each_entry_srcu'
- 'hlist_for_each_safe'
- 'hlist_nulls_for_each_entry'
- 'hlist_nulls_for_each_entry_from'
- 'hlist_nulls_for_each_entry_rcu'
- 'hlist_nulls_for_each_entry_safe'
- 'i3c_bus_for_each_i2cdev'
- 'i3c_bus_for_each_i3cdev'
- 'idr_for_each_entry'
- 'idr_for_each_entry_continue'
- 'idr_for_each_entry_continue_ul'
- 'idr_for_each_entry_ul'
- 'in_dev_for_each_ifa_rcu'
- 'in_dev_for_each_ifa_rtnl'
- 'inet_bind_bucket_for_each'
- 'interval_tree_for_each_span'
- 'intlist__for_each_entry'
- 'intlist__for_each_entry_safe'
- 'kcore_copy__for_each_phdr'
- 'key_for_each'
- 'key_for_each_safe'
- 'klp_for_each_func'
- 'klp_for_each_func_safe'
- 'klp_for_each_func_static'
- 'klp_for_each_object'
- 'klp_for_each_object_safe'
- 'klp_for_each_object_static'
- 'kunit_suite_for_each_test_case'
- 'kvm_for_each_memslot'
- 'kvm_for_each_memslot_in_gfn_range'
- 'kvm_for_each_vcpu'
- 'libbpf_nla_for_each_attr'
- 'list_for_each'
- 'list_for_each_codec'
- 'list_for_each_codec_safe'
- 'list_for_each_continue'
- 'list_for_each_entry'
- 'list_for_each_entry_continue'
- 'list_for_each_entry_continue_rcu'
- 'list_for_each_entry_continue_reverse'
- 'list_for_each_entry_from'
- 'list_for_each_entry_from_rcu'
- 'list_for_each_entry_from_reverse'
- 'list_for_each_entry_lockless'
- 'list_for_each_entry_rcu'
- 'list_for_each_entry_reverse'
- 'list_for_each_entry_safe'
- 'list_for_each_entry_safe_continue'
- 'list_for_each_entry_safe_from'
- 'list_for_each_entry_safe_reverse'
- 'list_for_each_entry_srcu'
- 'list_for_each_from'
- 'list_for_each_prev'
- 'list_for_each_prev_safe'
- 'list_for_each_rcu'
- 'list_for_each_reverse'
- 'list_for_each_safe'
- 'llist_for_each'
- 'llist_for_each_entry'
- 'llist_for_each_entry_safe'
- 'llist_for_each_safe'
- 'lwq_for_each_safe'
- 'map__for_each_symbol'
- 'map__for_each_symbol_by_name'
- 'maps__for_each_entry'
- 'maps__for_each_entry_safe'
- 'mas_for_each'
- 'mci_for_each_dimm'
- 'media_device_for_each_entity'
- 'media_device_for_each_intf'
- 'media_device_for_each_link'
- 'media_device_for_each_pad'
- 'media_entity_for_each_pad'
- 'media_pipeline_for_each_entity'
- 'media_pipeline_for_each_pad'
- 'mlx5_lag_for_each_peer_mdev'
- 'msi_domain_for_each_desc'
- 'msi_for_each_desc'
- 'mt_for_each'
- 'nanddev_io_for_each_page'
- 'netdev_for_each_lower_dev'
- 'netdev_for_each_lower_private'
- 'netdev_for_each_lower_private_rcu'
- 'netdev_for_each_mc_addr'
- 'netdev_for_each_synced_mc_addr'
- 'netdev_for_each_synced_uc_addr'
- 'netdev_for_each_uc_addr'
- 'netdev_for_each_upper_dev_rcu'
- 'netdev_hw_addr_list_for_each'
- 'nft_rule_for_each_expr'
- 'nla_for_each_attr'
- 'nla_for_each_nested'
- 'nlmsg_for_each_attr'
- 'nlmsg_for_each_msg'
- 'nr_neigh_for_each'
- 'nr_neigh_for_each_safe'
- 'nr_node_for_each'
- 'nr_node_for_each_safe'
- 'of_for_each_phandle'
- 'of_property_for_each_string'
- 'of_property_for_each_u32'
- 'pci_bus_for_each_resource'
- 'pci_dev_for_each_resource'
- 'pcl_for_each_chunk'
- 'pcl_for_each_segment'
- 'pcm_for_each_format'
- 'perf_config_items__for_each_entry'
- 'perf_config_sections__for_each_entry'
- 'perf_config_set__for_each_entry'
- 'perf_cpu_map__for_each_cpu'
- 'perf_cpu_map__for_each_idx'
- 'perf_evlist__for_each_entry'
- 'perf_evlist__for_each_entry_reverse'
- 'perf_evlist__for_each_entry_safe'
- 'perf_evlist__for_each_evsel'
- 'perf_evlist__for_each_mmap'
- 'perf_hpp_list__for_each_format'
- 'perf_hpp_list__for_each_format_safe'
- 'perf_hpp_list__for_each_sort_list'
- 'perf_hpp_list__for_each_sort_list_safe'
- 'perf_tool_event__for_each_event'
- 'plist_for_each'
- 'plist_for_each_continue'
- 'plist_for_each_entry'
- 'plist_for_each_entry_continue'
- 'plist_for_each_entry_safe'
- 'plist_for_each_safe'
- 'pnp_for_each_card'
- 'pnp_for_each_dev'
- 'protocol_for_each_card'
- 'protocol_for_each_dev'
- 'queue_for_each_hw_ctx'
- 'radix_tree_for_each_slot'
- 'radix_tree_for_each_tagged'
- 'rb_for_each'
- 'rbtree_postorder_for_each_entry_safe'
- 'rdma_for_each_block'
- 'rdma_for_each_port'
- 'rdma_umem_for_each_dma_block'
- 'resort_rb__for_each_entry'
- 'resource_list_for_each_entry'
- 'resource_list_for_each_entry_safe'
- 'rhl_for_each_entry_rcu'
- 'rhl_for_each_rcu'
- 'rht_for_each'
- 'rht_for_each_entry'
- 'rht_for_each_entry_from'
- 'rht_for_each_entry_rcu'
- 'rht_for_each_entry_rcu_from'
- 'rht_for_each_entry_safe'
- 'rht_for_each_from'
- 'rht_for_each_rcu'
- 'rht_for_each_rcu_from'
- 'rq_for_each_bvec'
- 'rq_for_each_segment'
- 'rq_list_for_each'
- 'rq_list_for_each_safe'
- 'sample_read_group__for_each'
- 'scsi_for_each_prot_sg'
- 'scsi_for_each_sg'
- 'sctp_for_each_hentry'
- 'sctp_skb_for_each'
- 'sec_for_each_insn'
- 'sec_for_each_insn_continue'
- 'sec_for_each_insn_from'
- 'sec_for_each_sym'
- 'shdma_for_each_chan'
- 'shost_for_each_device'
- 'sk_for_each'
- 'sk_for_each_bound'
- 'sk_for_each_bound_bhash2'
- 'sk_for_each_entry_offset_rcu'
- 'sk_for_each_from'
- 'sk_for_each_rcu'
- 'sk_for_each_safe'
- 'sk_nulls_for_each'
- 'sk_nulls_for_each_from'
- 'sk_nulls_for_each_rcu'
- 'snd_array_for_each'
- 'snd_pcm_group_for_each_entry'
- 'snd_soc_dapm_widget_for_each_path'
- 'snd_soc_dapm_widget_for_each_path_safe'
- 'snd_soc_dapm_widget_for_each_sink_path'
- 'snd_soc_dapm_widget_for_each_source_path'
- 'strlist__for_each_entry'
- 'strlist__for_each_entry_safe'
- 'sym_for_each_insn'
- 'sym_for_each_insn_continue_reverse'
- 'symbols__for_each_entry'
- 'tb_property_for_each'
- 'tcf_act_for_each_action'
- 'tcf_exts_for_each_action'
- 'ttm_resource_manager_for_each_res'
- 'twsk_for_each_bound_bhash2'
- 'udp_portaddr_for_each_entry'
- 'udp_portaddr_for_each_entry_rcu'
- 'usb_hub_for_each_child'
- 'v4l2_device_for_each_subdev'
- 'v4l2_m2m_for_each_dst_buf'
- 'v4l2_m2m_for_each_dst_buf_safe'
- 'v4l2_m2m_for_each_src_buf'
- 'v4l2_m2m_for_each_src_buf_safe'
- 'virtio_device_for_each_vq'
- 'while_for_each_ftrace_op'
- 'xa_for_each'
- 'xa_for_each_marked'
- 'xa_for_each_range'
- 'xa_for_each_start'
- 'xas_for_each'
- 'xas_for_each_conflict'
- 'xas_for_each_marked'
- 'xbc_array_for_each_value'
- 'xbc_for_each_key_value'
- 'xbc_node_for_each_array_value'
- 'xbc_node_for_each_child'
- 'xbc_node_for_each_key_value'
- 'xbc_node_for_each_subkey'
- 'zorro_for_each_dev'
- 'pool_iter_for_each'
- 'pool_for_each'
IncludeBlocks: Preserve
IncludeCategories:
- Regex: '.*'
Priority: 1
IncludeIsMainRegex: '(Test)?$'
IndentCaseLabels: false
IndentGotoLabels: false
IndentPPDirectives: None
IndentWidth: 8
IndentWrappedFunctionNames: false
JavaScriptQuotes: Leave
JavaScriptWrapImports: true
KeepEmptyLinesAtTheStartOfBlocks: false
MacroBlockBegin: ''
MacroBlockEnd: ''
MaxEmptyLinesToKeep: 1
NamespaceIndentation: None
ObjCBinPackProtocolList: Auto
ObjCBlockIndentWidth: 8
ObjCSpaceAfterProperty: true
ObjCSpaceBeforeProtocolList: true
# Taken from git's rules
PenaltyBreakAssignment: 10
PenaltyBreakBeforeFirstCallParameter: 30
PenaltyBreakComment: 10
PenaltyBreakFirstLessLess: 0
PenaltyBreakString: 10
PenaltyExcessCharacter: 100
PenaltyReturnTypeOnItsOwnLine: 60
PointerAlignment: Right
ReflowComments: false
SortIncludes: false
SortUsingDeclarations: false
SpaceAfterCStyleCast: false
SpaceAfterTemplateKeyword: true
SpaceBeforeAssignmentOperators: true
SpaceBeforeCtorInitializerColon: true
SpaceBeforeInheritanceColon: true
SpaceBeforeParens: ControlStatementsExceptForEachMacros
SpaceBeforeRangeBasedForLoopColon: true
SpaceInEmptyParentheses: false
SpacesBeforeTrailingComments: 1
SpacesInAngles: false
SpacesInContainerLiterals: false
SpacesInCStyleCastParentheses: false
SpacesInParentheses: false
SpacesInSquareBrackets: false
Standard: Cpp03
TabWidth: 8
UseTab: Always
...
================================================
FILE: .dockerignore
================================================
build
================================================
FILE: .github/workflows/build-freebsd.yml
================================================
name: build on FreeBSD
on:
push:
branches: [ "main", "dev" ]
pull_request:
branches: [ "main", "dev" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: apply the patch to libssh
run: |
git -C libssh fetch --all --tags --prune
patch -d libssh -p1 < patch/$(git -C libssh describe).patch
- name: Build in FreeBSD
uses: vmactions/freebsd-vm@v1
with:
prepare: |
pkg install -y git cmake
run: |
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build
build/mscp -h
================================================
FILE: .github/workflows/build-macos.yml
================================================
name: build on macOS
on:
push:
branches: [ "main", "dev" ]
pull_request:
branches: [ "main", "dev" ]
env:
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
BUILD_TYPE: Release
jobs:
build:
# The CMake configure and build commands are platform agnostic and should work equally well on Windows or Mac.
# You can convert this to a matrix build if you need cross-platform coverage.
# See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: apply the patch to libssh
run: |
git -C libssh fetch --all --tags --prune
patch -d libssh -p1 < patch/$(git -C libssh describe).patch
- name: install build dependency
run: ./scripts/install-build-deps.sh
- name: save homebrew prefix
id: brew-prefix
run: echo "HOMEBREW_PREFIX=$(brew --prefix)" >> $GITHUB_OUTPUT
- name: Configure CMake
# Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
# See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DOPENSSL_ROOT_DIR=${{steps.brew-prefix.outputs.HOMEBREW_PREFIX}}/opt/openssl@3
- name: Build
# Build your program with the given configuration
run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
- name: Run
run: ${{github.workspace}}/build/mscp -h
================================================
FILE: .github/workflows/build-ubuntu.yml
================================================
name: build on ubuntu
on:
push:
branches: [ "main", "dev" ]
pull_request:
branches: [ "main", "dev" ]
env:
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
BUILD_TYPE: Release
jobs:
build:
# The CMake configure and build commands are platform agnostic and should work equally well on Windows or Mac.
# You can convert this to a matrix build if you need cross-platform coverage.
# See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: apply the patch to libssh
run: |
git -C libssh fetch --all --tags --prune
patch -d libssh -p1 < patch/$(git -C libssh describe).patch
- name: install build dependency
run: |
sudo apt-get update
sudo ./scripts/install-build-deps.sh
- name: Configure CMake
# Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
# See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
- name: Build
# Build your program with the given configuration
run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
- name: Run
run: ${{github.workspace}}/build/mscp -h
================================================
FILE: .github/workflows/codeql.yml
================================================
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"
on:
push:
branches: [ "main", "dev" ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ "main", "dev" ]
schedule:
- cron: '35 11 * * 5'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'cpp' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
submodules: true
- name: apply the patch to libssh
run: |
git -C libssh fetch --all --tags --prune
patch -d libssh -p1 < patch/$(git -C libssh describe).patch
- name: install build dependency
run: |
sudo apt-get update
sudo ./scripts/install-build-deps.sh
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# Autobuild attempts to build any compiled languages (C/C++, C#, Go, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3
# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
================================================
FILE: .github/workflows/release.yml
================================================
name: release
on:
push:
tags:
- "v*.*.*"
workflow_dispatch:
env:
BUILD_TYPE: Release
jobs:
source-release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: apply the patch to libssh
run: |
git -C libssh fetch --all --tags --prune
patch -d libssh -p1 < patch/$(git -C libssh describe).patch
- name: Set variables
run: |
VER=$(cat VERSION)
echo "VERSION=$VER" >> $GITHUB_ENV
- name: archive
run: |
cd ..
cp -r mscp mscp-${{env.VERSION}}
tar zcvf mscp-${{env.VERSION}}.tar.gz --exclude-vcs mscp-${{env.VERSION}}
- name: Release
uses: softprops/action-gh-release@v1
with:
files: |
${{github.workspace}}/../mscp-${{env.VERSION}}.tar.gz
================================================
FILE: .github/workflows/test.yml
================================================
name: test
on:
push:
branches: [ "main", "dev" ]
pull_request:
branches: [ "main", "dev" ]
env:
BUILD_TYPE: Release
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
index: # see DIST_IDS and DIST_VERS lists in CMakeLists.txt
- ubuntu-20.04
- ubuntu-22.04
- ubuntu-24.04
- rocky-8.9
- rocky-9.3
- almalinux-9.3
- alpine-3.22
- arch-base
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: apply the patch to libssh
run: |
git -C libssh fetch --all --tags --prune
patch -d libssh -p1 < patch/$(git -C libssh describe).patch
# TODO: just building docker images does not require libssh. fix CMakeLists
- name: install build dependency
run: |
sudo apt-get update
sudo ./scripts/install-build-deps.sh
- name: configure CMake
run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
- name: Build Containers
run: make -C ${{github.workspace}}/build docker-build-${{ matrix.index }}
- name: Run Test
run: make -C ${{github.workspace}}/build docker-test-${{ matrix.index }}
================================================
FILE: .gitignore
================================================
build
html
compile_commands.json
CMakeUserPresets.json
.*.swp
.cache
include/mscp_version.h
================================================
FILE: .gitmodules
================================================
[submodule "libssh"]
path = libssh
url = https://git.libssh.org/projects/libssh.git
ignore = dirty
================================================
FILE: CMakeLists.txt
================================================
cmake_minimum_required(VERSION 3.13)
file (STRINGS "VERSION" MSCP_VERSION)
project(mscp
VERSION ${MSCP_VERSION}
LANGUAGES C)
find_package(Git)
if (Git_FOUND)
# based on https://github.com/nocnokneo/cmake-git-versioning-example
execute_process(
COMMAND ${GIT_EXECUTABLE} describe --tags --match "v*"
OUTPUT_VARIABLE GIT_DESCRIBE_VERSION
RESULT_VARIABLE GIT_DESCRIBE_ERROR_CODE
OUTPUT_STRIP_TRAILING_WHITESPACE)
if(NOT GIT_DESCRIBE_ERROR_CODE)
set(MSCP_BUILD_VERSION ${GIT_DESCRIBE_VERSION})
endif()
endif()
if (NOT MSCP_BUILD_VERSION)
message(STATUS "Failed to determine version via Git. Use VERSION file instead.")
set(MSCP_BUILD_VERSION v${MSCP_VERSION})
endif()
include(GNUInstallDirs)
set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -DDEBUG")
list(APPEND CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/cmake/modules)
if(APPLE)
list(APPEND CMAKE_PREFIX_PATH /usr/local) # intel mac homebrew prefix
list(APPEND CMAKE_PREFIX_PATH /opt/homebrew) # arm mac homebrew prefix
endif() # APPLE
option(BUILD_CONAN OFF) # Build mscp with conan
if(BUILD_CONAN)
message(STATUS "Build mscp with conan")
endif()
option(BUILD_STATIC OFF) # Build mscp with -static LD flag
if (BUILD_STATIC)
message(STATUS "Build mscp with -static LD option")
if (NOT BUILD_CONAN)
message(WARNING
"BUILD_STATIC strongly recommended with BUILD_CONAN option")
endif()
endif()
option(USE_PODMAN OFF) # use podman instread of docker
if(USE_PODMAN)
message(STATUS "Use podman instead of docker")
set(CE podman) # CE means Container Engine
else()
set(CE docker)
endif()
# add libssh static library
set(CMAKE_POLICY_DEFAULT_CMP0077 NEW)
set(WITH_SERVER OFF)
set(BUILD_SHARED_LIBS OFF)
set(WITH_EXAMPLES OFF)
set(BUILD_STATIC_LIB ON)
if(BUILD_CONAN)
message(STATUS
"Disable libssh GSSAPI support because libkrb5 doesn't exist in conan")
set(WITH_GSSAPI OFF)
endif()
add_subdirectory(libssh EXCLUDE_FROM_ALL)
# setup mscp compile options
list(APPEND MSCP_COMPILE_OPTS -iquote ${CMAKE_CURRENT_BINARY_DIR}/libssh/include)
list(APPEND MSCP_BUILD_INCLUDE_DIRS
${mscp_SOURCE_DIR}/src
${CMAKE_CURRENT_BINARY_DIR}/libssh/include)
list(APPEND MSCP_LINK_LIBS ssh-static)
if(BUILD_CONAN)
find_package(ZLIB REQUIRED)
find_package(OpenSSL REQUIRED)
list(APPEND MSCP_LINK_LIBS ZLIB::ZLIB)
list(APPEND MSCP_LINK_LIBS OpenSSL::Crypto)
endif()
# Symbol check
check_symbol_exists(htonll arpa/inet.h HAVE_HTONLL)
check_symbol_exists(ntohll arpa/inet.h HAVE_NTOHLL)
check_symbol_exists(strlcat string.h HAVE_STRLCAT)
if (NOT HAVE_STRLCAT)
list(APPEND OPENBSD_COMPAT_SRC src/openbsd-compat/strlcat.c)
endif()
# generate config.h in build dir
configure_file(
${mscp_SOURCE_DIR}/include/config.h.in
${CMAKE_CURRENT_BINARY_DIR}/include/config.h)
list(APPEND MSCP_BUILD_INCLUDE_DIRS ${CMAKE_CURRENT_BINARY_DIR}/include)
# libmscp.a
set(LIBMSCP_SRC
src/mscp.c src/ssh.c src/fileops.c src/path.c src/checkpoint.c
src/bwlimit.c src/platform.c src/print.c src/pool.c src/strerrno.c
${OPENBSD_COMPAT_SRC})
add_library(mscp-static STATIC ${LIBMSCP_SRC})
target_include_directories(mscp-static
PRIVATE ${MSCP_BUILD_INCLUDE_DIRS} ${mscp_SOURCE_DIR}/include)
target_compile_options(mscp-static PRIVATE ${MSCP_COMPILE_OPTS})
target_link_libraries(mscp-static PRIVATE ${MSCP_LINK_LIBS})
set_target_properties(mscp-static
PROPERTIES
OUTPUT_NAME mscp)
# mscp executable
list(APPEND MSCP_LINK_LIBS m pthread)
add_executable(mscp src/main.c)
target_include_directories(mscp
PRIVATE ${MSCP_BUILD_INCLUDE_DIRS} ${mscp_SOURCE_DIR}/include)
target_link_libraries(mscp mscp-static ${MSCP_LINK_LIBS})
if (BUILD_STATIC)
target_link_options(mscp PRIVATE -static)
endif()
target_compile_options(mscp PRIVATE ${MSCP_COMPILE_OPTS})
install(TARGETS mscp RUNTIME DESTINATION bin)
# mscp manpage and document
configure_file(
${mscp_SOURCE_DIR}/doc/mscp.1.in
${PROJECT_BINARY_DIR}/mscp.1)
add_custom_target(update-rst
COMMENT "Update doc/mscp.rst from mscp.1.in"
WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
COMMAND
pandoc -s -f man mscp.1 -t rst -o ${PROJECT_SOURCE_DIR}/doc/mscp.rst)
install(FILES ${PROJECT_BINARY_DIR}/mscp.1
DESTINATION ${CMAKE_INSTALL_MANDIR}/man1)
# Test
add_test(NAME pytest
COMMAND python3 -m pytest -v
--mscp-path=${PROJECT_BINARY_DIR}/mscp ${PROJECT_SOURCE_DIR}/test
WORKING_DIRECTORY ${PROJECT_BINARY_DIR})
enable_testing()
# Custom targets to build and test mscp in docker containers.
# foreach(IN ZIP_LISTS) (cmake >= 3.17) can shorten the following lists.
# However, ubuntu 20.04 has cmake 3.16.3. So this is a roundabout trick.
#
# When edit DIST_IDS and DIST_VERS, also edit .github/workflows/test.yaml
list(APPEND DIST_IDS ubuntu ubuntu ubuntu rocky rocky almalinux alpine arch)
list(APPEND DIST_VERS 20.04 22.04 24.04 8.9 9.3 9.3 3.22 base)
list(LENGTH DIST_IDS _DIST_LISTLEN)
math(EXPR DIST_LISTLEN "${_DIST_LISTLEN} - 1")
foreach(x RANGE ${DIST_LISTLEN})
list(GET DIST_IDS ${x} DIST_ID)
list(GET DIST_VERS ${x} DIST_VER)
set(DOCKER_IMAGE mscp-${DIST_ID}:${DIST_VER})
set(DOCKER_INDEX ${DIST_ID}-${DIST_VER})
execute_process(
COMMAND ${CMAKE_SOURCE_DIR}/scripts/install-build-deps.sh
--dont-install --platform Linux-${DIST_ID}
OUTPUT_VARIABLE REQUIREDPKGS
OUTPUT_STRIP_TRAILING_WHITESPACE)
add_custom_target(docker-build-${DOCKER_INDEX}
COMMENT "Build mscp in ${DOCKER_IMAGE} container"
WORKING_DIRECTORY ${mscp_SOURCE_DIR}
COMMAND
${CE} build --build-arg REQUIREDPKGS=${REQUIREDPKGS}
-t ${DOCKER_IMAGE} -f Dockerfile/${DOCKER_INDEX}.Dockerfile .)
add_custom_target(docker-build-${DOCKER_INDEX}-no-cache
COMMENT "Build mscp in ${DOCKER_IMAGE} container"
WORKING_DIRECTORY ${mscp_SOURCE_DIR}
COMMAND
${CE} build --build-arg REQUIREDPKGS=${REQUIREDPKGS} --no-cache
-t ${DOCKER_IMAGE} -f Dockerfile/${DOCKER_INDEX}.Dockerfile .)
add_custom_target(docker-test-${DOCKER_INDEX}
COMMENT "Test mscp in ${DOCKER_IMAGE} container"
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
COMMAND
${CE} run --init --rm --privileged
--sysctl net.ipv6.conf.all.disable_ipv6=0
--add-host=ip6-localhost:::1
${DOCKER_IMAGE} /mscp/scripts/test-in-container.sh)
add_custom_target(docker-run-${DOCKER_INDEX}
COMMENT "Start ${DOCKER_IMAGE} container"
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
COMMAND
${CE} run --init --rm --privileged
--sysctl net.ipv6.conf.all.disable_ipv6=0
--add-host=ip6-localhost:::1
-it
${DOCKER_IMAGE} /mscp/scripts/test-in-container.sh bash)
list(APPEND DOCKER_BUILDS docker-build-${DOCKER_INDEX})
list(APPEND DOCKER_BUILDS_NO_CACHE docker-build-${DOCKER_INDEX}-no-cache)
list(APPEND DOCKER_TESTS docker-test-${DOCKER_INDEX})
endforeach()
add_custom_target(docker-build-all DEPENDS ${DOCKER_BUILDS})
add_custom_target(docker-build-all-no-cache DEPENDS ${DOCKER_BUILDS_NO_CACHE})
add_custom_target(docker-test-all DEPENDS ${DOCKER_TESTS})
### debuild-related definitions
set(DEBBUILDCONTAINER mscp-build-deb)
execute_process(
COMMAND ${CMAKE_SOURCE_DIR}/scripts/install-build-deps.sh
--dont-install --platform Linux-ubuntu
OUTPUT_VARIABLE REQUIREDPKGS_DEB
OUTPUT_STRIP_TRAILING_WHITESPACE)
add_custom_target(build-deb
COMMENT "build mscp deb files inside a container"
WORKING_DIRECTORY ${mscp_SOURCE_DIR}
BYPRODUCTS ${CMAKE_BINARY_DIR}/debbuild
COMMAND
${CE} build --build-arg REQUIREDPKGS=${REQUIREDPKGS_DEB}
-t ${DEBBUILDCONTAINER} -f Dockerfile/build-deb.Dockerfile .
COMMAND
${CE} run --rm -v ${CMAKE_BINARY_DIR}:/out ${DEBBUILDCONTAINER}
cp -r /debbuild /out/)
### rpmbuild-related definitions
# generate files for rpmbuild
configure_file(
${mscp_SOURCE_DIR}/rpm/mscp.spec.in
${mscp_SOURCE_DIR}/rpm/mscp.spec
@ONLY)
#configure_file(
# ${mscp_SOURCE_DIR}/Dockerfile/build-srpm.Dockerfile.in
# ${mscp_SOURCE_DIR}/Dockerfile/build-srpm.Dockerfile
# @ONLY)
# Custom target to build mscp as a src.rpm in docker.
set(RPMBUILDCONTAINER mscp-build-srpm)
execute_process(
COMMAND ${CMAKE_SOURCE_DIR}/scripts/install-build-deps.sh
--dont-install --platform Linux-rocky
OUTPUT_VARIABLE REQUIREDPKGS_RPM
OUTPUT_STRIP_TRAILING_WHITESPACE)
add_custom_target(build-srpm
COMMENT "Build mscp src.rpm inside a container"
WORKING_DIRECTORY ${mscp_SOURCE_DIR}
COMMAND
${CE} build --build-arg REQUIREDPKGS=${REQUIREDPKGS_RPM}
--build-arg MSCP_VERSION=${MSCP_VERSION}
-t ${RPMBUILDCONTAINER} -f Dockerfile/build-srpm.Dockerfile .
COMMAND
${CE} run --rm -v ${CMAKE_BINARY_DIR}:/out ${RPMBUILDCONTAINER}
bash -c "cp /root/rpmbuild/SRPMS/mscp-*.src.rpm /out/")
### single-binary-build-related definitions
# Custom target to get single binary mscp
set(SINGLEBINARYFILE mscp.linux.${CMAKE_SYSTEM_PROCESSOR}.static)
add_custom_target(build-single-binary
COMMENT "Build mscp as a single binary in alpine conatiner"
WORKING_DIRECTORY ${mscp_SOURCE_DIR}
BYPRODUCTS ${CMAKE_BINARY_DIR}/${SINGLEBINARYFILE}
DEPENDS docker-build-alpine-3.22
COMMAND
${CE} run --rm -v ${CMAKE_BINARY_DIR}:/out mscp-alpine:3.22
cp /mscp/build/mscp /out/${SINGLEBINARYFILE})
add_custom_target(build-pkg-all
DEPENDS build-deb build-srpm build-single-binary)
================================================
FILE: Dockerfile/README.md
================================================
Dockerfiles for building and testing mscp.
cmake provides custom targets to build and test mscp in the containers
See `make docker-*` targets. `make docker-build-all` builds all
container images, and `make docker-test-all` runs the test in all
container images.
================================================
FILE: Dockerfile/almalinux-9.3.Dockerfile
================================================
FROM almalinux:9.3
ARG REQUIREDPKGS
# install pytest, sshd for test, and rpm-build
RUN set -ex && yum -y install \
${REQUIREDPKGS} python3 python3-pip python3-devel \
openssh openssh-server openssh-clients rpm-build
RUN python3 -m pip install pytest
# preparation for sshd
RUN mkdir /var/run/sshd \
&& ssh-keygen -A \
&& ssh-keygen -f /root/.ssh/id_rsa -N "" \
&& cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
# create test user
RUN useradd -m -d /home/test test \
&& echo "test:userpassword" | chpasswd \
&& mkdir -p /home/test/.ssh \
&& ssh-keygen -f /home/test/.ssh/id_rsa_test -N "keypassphrase" \
&& cat /home/test/.ssh/id_rsa_test.pub >> /home/test/.ssh/authorized_keys \
&& chown -R test:test /home/test \
&& chown -R test:test /home/test/.ssh
RUN rm -rf /run/nologin
ARG mscpdir="/mscp"
COPY . ${mscpdir}
# build
RUN cd ${mscpdir} \
&& rm -rf build \
&& cmake -B build \
&& cd ${mscpdir}/build \
&& make -j 2 \
&& make install
================================================
FILE: Dockerfile/alpine-3.22.Dockerfile
================================================
FROM alpine:3.22
# do not use REQUIREDPKGS build argument because
# this Dockerfile compiles mscp with conan,so we do not need
# libssl-dev and zlib-dev
# Build mscp with conan to create single binary mscp
RUN apk add --no-cache \
gcc make cmake libc-dev \
linux-headers openssh bash perl \
python3 py3-pip python3-dev py3-pytest g++
RUN pip3 install --break-system-packages conan
# preparation for sshd
RUN ssh-keygen -A \
&& mkdir /var/run/sshd \
&& ssh-keygen -f /root/.ssh/id_rsa -N "" \
&& cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
# disable PerSourcePenaltie, which would distrub test:
# https://undeadly.org/cgi?action=article;sid=20240607042157
RUN echo "PerSourcePenalties=no" > /etc/ssh/sshd_config.d/90-mscp-test.conf
# create test user
RUN addgroup -S test \
&& adduser -S test -G test \
&& echo "test:userpassword" | chpasswd \
&& mkdir -p /home/test/.ssh \
&& ssh-keygen -f /home/test/.ssh/id_rsa_test -N "keypassphrase" \
&& cat /home/test/.ssh/id_rsa_test.pub >> /home/test/.ssh/authorized_keys \
&& chown -R test:test /home/test \
&& chown -R test:test /home/test/.ssh
# Build mscp as a single binary
RUN conan profile detect --force
ARG mscpdir="/mscp"
COPY . ${mscpdir}
RUN cd ${mscpdir} \
&& rm -rf build \
&& conan install . --output-folder=build --build=missing \
&& cd ${mscpdir}/build \
&& cmake .. \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake \
-DBUILD_CONAN=ON -DBUILD_STATIC=ON \
&& make -j 2 \
&& make install
================================================
FILE: Dockerfile/arch-base.Dockerfile
================================================
FROM archlinux:base
ARG REQUIREDPKGS
# install pyest and openssh for test
RUN set -ex && pacman -Syy && pacman --noconfirm -S ${REQUIREDPKGS} openssh python-pytest
RUN mkdir /var/run/sshd \
&& ssh-keygen -A \
&& ssh-keygen -f /root/.ssh/id_rsa -N "" \
&& cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
# disable PerSourcePenaltie, which would distrub test:
# https://undeadly.org/cgi?action=article;sid=20240607042157
RUN echo "PerSourcePenalties=no" > /etc/ssh/sshd_config.d/90-mscp-test.conf
# create test user
RUN useradd -m -d /home/test test \
&& echo "test:userpassword" | chpasswd \
&& mkdir -p /home/test/.ssh \
&& ssh-keygen -f /home/test/.ssh/id_rsa_test -N "keypassphrase" \
&& cat /home/test/.ssh/id_rsa_test.pub >> /home/test/.ssh/authorized_keys \
&& chown -R test:test /home/test \
&& chown -R test:test /home/test/.ssh
ARG mscpdir="/mscp"
COPY . ${mscpdir}
# build
RUN cd ${mscpdir} \
&& rm -rf build \
&& cmake -B build \
&& cd ${mscpdir}/build \
&& make -j 2 \
&& make install
================================================
FILE: Dockerfile/build-deb.Dockerfile
================================================
FROM ubuntu:22.04
ARG REQUIREDPKGS
ARG DEBIAN_FRONTEND=noninteractive
RUN set -ex && apt-get update && apt-get install -y --no-install-recommends \
${REQUIREDPKGS} ca-certificates \
build-essential devscripts debhelper gcc make cmake
ARG mscpdir="/debbuild/mscp"
COPY . ${mscpdir}
# build
RUN cd ${mscpdir} \
&& debuild -us -uc -S \
&& mv ${mscpdir} /
# Then all debuild output files exsit at /debbuild
================================================
FILE: Dockerfile/build-srpm.Dockerfile
================================================
FROM rockylinux:9
ARG REQUIREDPKGS
ARG MSCP_VERSION
# install pytest, sshd for test, and rpm-build
RUN set -ex && yum -y install ${REQUIREDPKGS} rpm-build rpmdevtools
ARG mscpdir="/mscp-${MSCP_VERSION}"
ARG mscptgz="mscp-${MSCP_VERSION}.tar.gz"
COPY . ${mscpdir}
# prepare rpmbuild
RUN rpmdev-setuptree \
&& rm -rf ${mscpdir}/build \
&& tar zcvf /${mscptgz} --exclude-vcs ${mscpdir} \
&& cp /${mscptgz} ~/rpmbuild/SOURCES/ \
&& cp ${mscpdir}/rpm/mscp.spec ~/rpmbuild/SPECS/
# build rpm and src.rpm
RUN rpmbuild -ba ~/rpmbuild/SPECS/mscp.spec
================================================
FILE: Dockerfile/rocky-8.9.Dockerfile
================================================
FROM rockylinux:8.9
ARG REQUIREDPKGS
# install pytest, sshd for test, and rpm-build
RUN set -ex && yum -y install \
${REQUIREDPKGS} \
python3 python3-pip python3-devel \
openssh openssh-server openssh-clients rpm-build
RUN python3 -m pip install pytest
# preparation for sshd
RUN mkdir /var/run/sshd \
&& ssh-keygen -A \
&& ssh-keygen -f /root/.ssh/id_rsa -N "" \
&& cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
# create test user
RUN useradd -m -d /home/test test \
&& echo "test:userpassword" | chpasswd \
&& mkdir -p /home/test/.ssh \
&& ssh-keygen -f /home/test/.ssh/id_rsa_test -N "keypassphrase" \
&& cat /home/test/.ssh/id_rsa_test.pub >> /home/test/.ssh/authorized_keys \
&& chown -R test:test /home/test \
&& chown -R test:test /home/test/.ssh
RUN rm -rf /run/nologin
ARG mscpdir="/mscp"
COPY . ${mscpdir}
# build
RUN cd ${mscpdir} \
&& rm -rf build \
&& cmake -B build \
&& cd ${mscpdir}/build \
&& make -j 2 \
&& make install
================================================
FILE: Dockerfile/rocky-9.3.Dockerfile
================================================
FROM rockylinux:9.3
ARG REQUIREDPKGS
# install pytest, sshd for test, and rpm-build
RUN set -ex && yum -y install \
${REQUIREDPKGS} \
python3 python3-pip python3-devel \
openssh openssh-server openssh-clients rpm-build
RUN python3 -m pip install pytest
# preparation for sshd
RUN mkdir /var/run/sshd \
&& ssh-keygen -A \
&& ssh-keygen -f /root/.ssh/id_rsa -N "" \
&& cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
# create test user
RUN useradd -m -d /home/test test \
&& echo "test:userpassword" | chpasswd \
&& mkdir -p /home/test/.ssh \
&& ssh-keygen -f /home/test/.ssh/id_rsa_test -N "keypassphrase" \
&& cat /home/test/.ssh/id_rsa_test.pub >> /home/test/.ssh/authorized_keys \
&& chown -R test:test /home/test \
&& chown -R test:test /home/test/.ssh
RUN rm -rf /run/nologin
ARG mscpdir="/mscp"
COPY . ${mscpdir}
# build
RUN cd ${mscpdir} \
&& rm -rf build \
&& cmake -B build \
&& cd ${mscpdir}/build \
&& make -j 2 \
&& make install
================================================
FILE: Dockerfile/ubuntu-20.04.Dockerfile
================================================
FROM ubuntu:20.04
ARG REQUIREDPKGS
ARG DEBIAN_FRONTEND=noninteractive
RUN set -ex && apt-get update && apt-get install -y --no-install-recommends \
${REQUIREDPKGS} ca-certificates python3 python3-pip python3-dev openssh-server
RUN python3 -m pip install pytest
# preparation for sshd
RUN mkdir /var/run/sshd \
&& ssh-keygen -A \
&& ssh-keygen -f /root/.ssh/id_rsa -N "" \
&& cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
# create test user
RUN useradd -m -d /home/test test \
&& echo "test:userpassword" | chpasswd \
&& mkdir -p /home/test/.ssh \
&& ssh-keygen -f /home/test/.ssh/id_rsa_test -N "keypassphrase" \
&& cat /home/test/.ssh/id_rsa_test.pub >> /home/test/.ssh/authorized_keys \
&& chown -R test:test /home/test \
&& chown -R test:test /home/test/.ssh
ARG mscpdir="/mscp"
COPY . ${mscpdir}
# build
RUN cd ${mscpdir} \
&& rm -rf build \
&& cmake -B build \
&& cd ${mscpdir}/build \
&& make -j 2 \
&& make install
================================================
FILE: Dockerfile/ubuntu-22.04.Dockerfile
================================================
FROM ubuntu:22.04
ARG REQUIREDPKGS
ARG DEBIAN_FRONTEND=noninteractive
RUN set -ex && apt-get update && apt-get install -y --no-install-recommends \
${REQUIREDPKGS} ca-certificates python3 python3-pip python3-dev openssh-server
RUN python3 -m pip install pytest
# preparation for sshd
RUN mkdir /var/run/sshd \
&& ssh-keygen -A \
&& ssh-keygen -f /root/.ssh/id_rsa -N "" \
&& cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
# create test user
RUN useradd -m -d /home/test test \
&& echo "test:userpassword" | chpasswd \
&& mkdir -p /home/test/.ssh \
&& ssh-keygen -f /home/test/.ssh/id_rsa_test -N "keypassphrase" \
&& cat /home/test/.ssh/id_rsa_test.pub >> /home/test/.ssh/authorized_keys \
&& chown -R test:test /home/test \
&& chown -R test:test /home/test/.ssh
ARG mscpdir="/mscp"
COPY . ${mscpdir}
# build
RUN cd ${mscpdir} \
&& rm -rf build \
&& cmake -B build \
&& cd ${mscpdir}/build \
&& make -j 2 \
&& make install
================================================
FILE: Dockerfile/ubuntu-24.04.Dockerfile
================================================
FROM ubuntu:24.04
ARG REQUIREDPKGS
ARG DEBIAN_FRONTEND=noninteractive
RUN set -ex && apt-get update && apt-get install -y --no-install-recommends \
${REQUIREDPKGS} ca-certificates openssh-server vim-tiny \
python3 python3-pip python3-dev python3-pytest
# preparation for sshd
RUN mkdir /var/run/sshd \
&& ssh-keygen -A \
&& ssh-keygen -f /root/.ssh/id_rsa -N "" \
&& cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
# create test user
RUN useradd -m -d /home/test test \
&& echo "test:userpassword" | chpasswd \
&& mkdir -p /home/test/.ssh \
&& ssh-keygen -f /home/test/.ssh/id_rsa_test -N "keypassphrase" \
&& cat /home/test/.ssh/id_rsa_test.pub >> /home/test/.ssh/authorized_keys \
&& chown -R test:test /home/test \
&& chown -R test:test /home/test/.ssh
ARG mscpdir="/mscp"
COPY . ${mscpdir}
# build
RUN cd ${mscpdir} \
&& rm -rf build \
&& cmake -B build \
&& cd ${mscpdir}/build \
&& make -j 2 \
&& make install
================================================
FILE: Doxyfile
================================================
PROJECT_NAME = libmscp
GENERATE_HTML = YES
GENERATE_CHI = NO
GENERATE_LATEX = NO
GENERATE_RTF = NO
GENERATE_MAN = NO
SOURCE_BROWSER = YES
INPUT = src include
================================================
FILE: LICENSE
================================================
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.
================================================
FILE: README.md
================================================
# mscp: multi-threaded scp
[](https://github.com/upa/mscp/actions/workflows/build-ubuntu.yml)
[](https://github.com/upa/mscp/actions/workflows/build-macos.yml)
[](https://github.com/upa/mscp/actions/workflows/build-freebsd.yml)
[](https://github.com/upa/mscp/actions/workflows/test.yml)
`mscp`, a variant of `scp`, copies files over multiple SSH (SFTP)
connections by multiple threads. It enables transferring (1) multiple
files simultaneously and (2) a large file in parallel, reducing the
transfer time for a lot of/large files over networks.
You can use `mscp` like `scp`, for example:
```shell-session
$ mscp srcfile user@example.com:dstfile
```
Remote hosts only need to run standard `sshd` supporting the SFTP
subsystem (e.g. openssh-server), and you need to be able to ssh to the
hosts as usual. `mscp` does not require anything else.
https://github.com/upa/mscp/assets/184632/19230f57-be7f-4ef0-98dd-cb4c460f570d
--------------------------------------------------------------------
Major differences from `scp` on usage:
- Remote-to-remote copy is not supported.
- `-r` option is not needed to transfer directories.
- Checkpointing for resuming failed transfer is supported.
- and any other differences I have not implemented and noticed.
Paper:
- Ryo Nakamura and Yohei Kuga. 2023. Multi-threaded scp: Easy and Fast File Transfer over SSH. In Practice and Experience in Advanced Research Computing (PEARC '23). Association for Computing Machinery, New York, NY, USA, 320–323. https://doi.org/10.1145/3569951.3597582
## Install
- macOS
```console
# Homebrew
brew install upa/tap/mscp
# MacPorts
sudo port install mscp
```
- Ubuntu
```console
sudo add-apt-repository ppa:upaa/mscp
sudo apt-get install mscp
```
- RHEL-based distributions
```console
sudo dnf copr enable upaaa/mscp
sudo dnf install mscp
```
## Build
mscp depends on a patched [libssh](https://www.libssh.org/). The
patch introduces asynchronous SFTP Write, which is derived from
https://github.com/limes-datentechnik-gmbh/libssh (see [Re: SFTP Write
async](https://archive.libssh.org/libssh/2020-06/0000004.html)).
We test building mscp on Linux (Ubuntu, Rocky, Alma, and Alpine),
macOS, and FreeBSD.
```console
# clone this repository
git clone https://github.com/upa/mscp.git
cd mscp
# prepare patched libssh
git submodule update --init
patch -d libssh -p1 < patch/$(git -C libssh describe).patch
# install build dependency
bash ./scripts/install-build-deps.sh
# configure mscp
mkdir build && cd build
cmake ..
# in macOS, you may need OPENSSL_ROOT_DIR for cmake:
# cmake .. -DOPENSSL_ROOT_DIR=$(brew --prefix)/opt/openssl@3
# build
make
# install the mscp binary to CMAKE_INSTALL_PREFIX/bin (usually /usr/local/bin)
make install
```
Source tar balls (`mscp-X.X.X.tar.gz`, not `Source code`) in
[Releases page](https://github.com/upa/mscp/releases) contain the patched version
of libssh. So you can start from cmake with it.
## Documentation
[manpage](/doc/mscp.rst) is available.
================================================
FILE: VERSION
================================================
0.2.4
================================================
FILE: conanfile.txt
================================================
[requires]
zlib/1.2.11
openssl/1.1.1t
[generators]
CMakeDeps
CMakeToolchain
================================================
FILE: doc/RELEASE.md
================================================
## Build mscp as deb package
`make build-deb` produces a mscp deb package and related files. This
target builds mscp with `debuild` inside a docker container
(Dockerfile is `docker/build-deb.Docerfile`).
```console
mkdir build && cd build && cmake ..
make build-deb
```
After that:
```console
$ ls debbuild
mscp_0.1.4.dsc mscp_0.1.4_source.buildinfo mscp_0.1.4.tar.xz
mscp_0.1.4_source.build mscp_0.1.4_source.changes
```
### To publush mscp in launchpad PPA:
1. write changes in `debian/changelog` at main branch (the date
command needed here is `date -R`)
2. switch to `ppa-focal` or `ppa-jammy` branch
3. rebase to the `main` branch and modify `debian/changes`:
* change `mscp (X.X.X) UNRELEASED;` to `mscp (X.X.X-1~RELEASENAME) RELEASENAME;`
where `RELEASENAME` is `focal` or `jammy`.
4. run `make build-deb` at the build directory and `cd debbuild`
5. sign the files with `debsign -k [GPGKEYID] mscp_X.X.X~X_source.changes`
5. upload the files with `dput ppa:upaa/mscp mscp_X.X.X~X_source.changes`
## Build mscp as (source) rpm package
`make build-srpm` produces a mscp src.rpm package. This target builts
mscp with `rpmbuild` inside a docker container (Dockerfile is
`docker/build-srpm.Dockerfile`, generated from
`build-srpm.Dockerfile.in` by cmake).
```console
mkdir build && cd build && cmake ..
make build-srpm
```
After that:
```console
$ ls *.rpm
mscp-0.1.3-1.el9.src.rpm
```
### To publish mscp in COPR:
1. update `rpm/mscp.spec.in`, the `changelog` section (the date
command needed here is `date "+%a %b %d %Y"`)
2. run `make build-srpm`
3. download `mscp-X.X.X-1.yyy.src.rpm`
4. upload the src.rpm to Build page at COPR.
## Update Document
The docuemnt is `doc/mscp.rst` (at present). When `mscp.1.in` is
modified, run `make update-rst` to make it up to date.
```console
mkdir build cd build && cmake ..
make update-rst
```
================================================
FILE: doc/mscp.1.in
================================================
.TH MSCP 1 "@MSCP_BUILD_VERSION@" "mscp" "User Commands"
.SH NAME
mscp \- copy files over multiple SSH connections
.SH SYNOPSIS
.B mscp
.RB [ \-46vqDpdNh ]
[\c
.BI \-n \ NR_CONNECTIONS\c
]
[\c
.BI \-m \ COREMASK\c
]
[\c
.BI \-u \ MAX_STARTUPS\c
]
[\c
.BI \-I \ INTERVAL\c
]
[\c
.BI \-W \ CHECKPOINT\c
]
[\c
.BI \-R \ CHECKPOINT\c
]
[\c
.BI \-s \ MIN_CHUNK_SIZE\c
]
[\c
.BI \-S \ MAX_CHUNK_SIZE\c
]
[\c
.BI \-a \ NR_AHEAD\c
]
[\c
.BI \-b \ BUF_SIZE\c
]
[\c
.BI \-L \ LIMIT_BITRATE\c
]
[\c
.BI \-l \ LOGIN_NAME\c
]
[\c
.BI \-P \ PORT\c
]
[\c
.BI \-F \ SSH_CONFIG\c
]
[\c
.BI \-o \ SSH_OPTION\c
]
[\c
.BI \-i \ IDENTITY\c
]
[\c
.BI \-J \ DESTINATION\c
]
[\c
.BI \-c \ CIPHER\c
]
[\c
.BI \-M \ HMAC\c
]
[\c
.BI \-C \ COMPRESS\c
]
[\c
.BI \-g \ CONGESTION\c
]
.I source ... target
.SH DESCRIPTION
.PP
.B mscp
copies files over multiple SSH (SFTP) connections by multiple
threads. It enables transferring (1) multiple files simultaneously and
(2) a large file in parallel, reducing the transfer time for a lot
of/large files over networks.
.PP
The usage of
.B mscp
follows the
.B scp
command of
.I OpenSSH,
for example:
.nf
$ mscp srcfile user@example.com:dstfile
.fi
Remote hosts only need to run standard
.B sshd
supporting the SFTP subsystem, and users need to be able to
.B ssh
to the hosts as usual.
.B mscp
does not require anything else.
.PP
.B mscp
uses
.UR https://\:www\:.libssh\:.org
libssh
.UE
as its SSH implementation. Thus, supported SSH features, for example,
authentication, encryption, and various options in ssh_config, follow
what
.I libssh
supports.
.SH OPTIONS
.TP
.B \-n \fINR_CONNECTIONS\fR
Specifies the number of SSH connections. The default value is
calculated from the number of CPU cores on the host with the following
formula: floor(log(nr_cores)*2)+1.
.TP
.B \-m \fICOREMASK\fR
Configures CPU cores to be used by the hexadecimal bitmask. For
example, -m 0x25 pins threads onto CPU cores 0, 2, and 5. The default
value is not specified: all CPU cores are used and no threads are
pinned to any cores.
.TP
.B \-u \fIMAX_STARTUPS\fR
Specifies the number of concurrent unauthenticated SSH connection
attempts.
.B sshd
limits the number of simultaneous SSH connection attempts by
.I MaxStartups
in
.I sshd_config.
The default
.I MaxStartups
is 10; thus, we set the default MAX_STARTUPS 8.
.TP
.B \-I \fIINTERVAL\fR
Specifies the interval (in seconds) between SSH connection
attempts. Some firewall products treat SSH connection attempts from a
single source IP address for a short period as a brute force attack.
This option inserts intervals between the attempts to avoid being
determined as an attack. The default value is 0.
.TP
.B \-W \fICHECKPOINT\fR
Specifies a checkpoint file to save the state of a failed
transfer. When transferring fails due to, for example, connection
disruption or user interrupt,
.B mscp
writes the information about the remaining files and chunks to the
specified checkpoint file.
.B \-W
option with
.B \-D
(dry-run mode) only writes a checkpoint file and exits.
.TP
.B \-R \fICHECKPOINT\fR
Specifies a checkpoint file to resume a transfer. When a checkpoint
file is passed,
.B mscp
reads the checkpoint to load a remote host, copy direction, and files
and their chunks to be transferred. Namely,
.B mscp
can resume a past failed transfer from the checkpoint. Resuming with a
checkpoint does not require
.I source ... target
arguments. Other SSH connection options, such as port number and
config file, should be specified as with the failed run. In addition,
checkpoint files have file paths as relative paths. Thus, you must run
.B mscp
in the same working directory as the failed run. You can see the
contents of a checkpoint file with the
.B mscp \-vv \-D \-R CHECKPOINT
command (Dry-run mode). Note that the checkpoint file is not
automatically removed after the resumed transfer ends
successfully. Users should check the return value of
.B mscp
and remove the checkpoint if it returns 0.
.TP
.B \-s \fIMIN_CHUNK_SIZE\fR
Specifies the minimum chunk size.
.B mscp
divides a single file into chunks and copies the chunks in
parallel. The default value is 16M bytes.
.TP
.B \-S \fIMAX_CHUNK_SIZE\fR
Specifies the maximum chunk size. The default is file size divided by
the number of connections and devided by 4. If the calculated value
is smarller than the
.B MIN_CHUNK_SIZE
value,
MIN_CHUNK_SIZE is used.
.TP
.B \-a \fINR_AHEAD\fR
Specifies the number of inflight SFTP commands. The default value is
32.
.TP
.B \-b \fIBUF_SIZE\fR
Specifies the buffer size for I/O and transfer over SFTP. The default
value is 16384. Note that the SSH specification restricts buffer size
delivered over SSH. Changing this value is not recommended at present.
.TP
.B \-L \fILIMIT_BITRATE\fR
Limits the bitrate, specified with k (K), m (M), and g (G), e.g., 100m
indicates 100 Mbps.
.TP
.B \-4
Uses IPv4 addresses only.
.TP
.B \-6
Uses IPv6 addresses only.
.TP
.B \-v
Increments the verbose output level.
.TP
.B \-q
Quiet mode: turns off all outputs.
.TP
.B \-D
Dry-run mode: it scans source files to be copied, calculates chunks,
resolves destination file paths, and exits. Dry-run mode with
.B -vv
option can confirm files to be copied and their destination paths.
.TP
.B \-r
No effect.
.B mscp
copies recursively if a source path is a directory. This option exists
for just compatibility.
.TP
.B \-l \fILOGIN_NAME\fR
Specifies the username to log in on the remote machine as with
.I ssh(1).
.TP
.B \-P \fIPORT\fR
Specifies the port number to connect to on the remote machine as with
.I scp(1).
.TP
.B \-F \fISSH_CONFIG\fR
Specifies an alternative per-user ssh configuration file. Note that
acceptable options in the configuration file are what
.I libssh
supports.
.TP
.B \-o \fISSH_OPTION\fR
Specifies ssh options in the format used in ssh_config. Note that
acceptable options are what
.I libssh
supports.
.TP
.B \-i \fIIDENTITY\fR
Specifies the identity file for public key authentication.
.TP
.B \-J \fIDESTINATION\fR
A shortcut to define a
.B ProxyJump
configuration directive. Each SFTP session of
.B mscp
connects to the target host by first making an
.B ssh
connection to the jump host described by
.I destination.
.TP
.B \-c \fICIPHER\fR
Selects the cipher to use for encrypting the data transfer. See
.B mscp -h
or
.B Ciphers
in
.UR https://\:www\:.libssh\:.org/\:features/
libssh features
.UE .
.TP
.B \-M \fIHMAC\fR
Specifies MAC hash algorithms. See
.B mscp -h
or
.B MAC hashes
in
.UR https://\:www\:.libssh\:.org/\:features/
libssh features
.UE .
.TP
.B \-C \fICOMPRESS\fR
Enables compression: yes, no, zlib, zlib@openssh.com. The default is
none. See
.UR https://\:www\:.libssh\:.org/\:features/
libssh features
.UE .
.TP
.B \-g \fICONGESTION\fR
Specifies the TCP congestion control algorithm to use (Linux only).
See
.B sysctl net.ipv4.tcp_allowed_congestion_control
for available values.
.TP
.B \-p
Preserves modification times and access times (file mode bits are
preserved by default).
.TP
.B \-d
Increments the ssh debug output level.
.TP
.B \-N
Enables Nagle's algorithm. It is disabled by default.
.TP
.B \-h
Prints help.
.SH EXIT STATUS
Exit status is 0 on success, and >0 if an error occurs.
.SH ENVIRONMENT
.PP
.B mscp
recognizes the following environment variables.
.TP
.B MSCP_SSH_AUTH_PASSWORD
This environment variable passes a password for password
authentication to establish SSH connections.
.TP
.B MSCP_SSH_AUTH_PASSPHRASE
This environment variable passes a passphrase for public-key
authentication for establishing SSH connections.
.SH NOTES
.PP
.B mscp
uses glob(3) for globbing pathnames, including matching patterns for
local and remote paths. However, globbing on the
.I remote
side does not work with musl libc (used in Alpine Linux and the
single-binary version of mscp) because musl libc does not support
GLOB_ALTDIRFUNC.
.PP
.B mscp
does not support remote-to-remote copy, which
.B scp
supports.
.SH EXAMPLES
.PP
Copy a local file to a remote host with different name:
.nf
$ mscp ~/src-file 10.0.0.1:copied-file
.fi
.PP
Copy a local file and a directory to /tmp at a remote host:
.nf
$ mscp ~/src-file dir1 10.0.0.1:/tmp
.fi
.PP
Save a checkpoint if transfer fails:
.nf
$ mscp -W mscp.checkpoint many-large-files 10.0.0.1:dst/
.fi
.PP
Check the remaining files and chunks, and resume the failed transfer:
.nf
# Dump the content of a checkpoint and exit (dry-run mode)
$ mscp -vv -D -R mscp.checkpoint
# resume transferring from the checkpoint
$ mscp -R mscp.checkpoint
.fi
.PP
In a long fat network, following options might improve performance:
.nf
$ mscp -n 64 -m 0xffff -a 64 -c aes128-gcm@openssh.com src 10.0.0.1:
.fi
.B -n
increases the number of SSH connections than default,
.B -m
pins threads to specific CPU cores,
.B -a
increases asynchronous inflight SFTP WRITE/READ commands, and
.B -c aes128-gcm@openssh.com
will be faster than the default chacha20-poly1305 cipher, particularly
on hosts that support AES-NI.
.SH "SEE ALSO"
.BR scp (1),
.BR ssh (1),
.BR sshd (8).
.SH "PAPER REFERENCE"
Ryo Nakamura and Yohei Kuga. 2023. Multi-threaded scp: Easy and Fast
File Transfer over SSH. In Practice and Experience in Advanced
Research Computing (PEARC '23). Association for Computing Machinery,
New York, NY, USA, 320–323.
.UR https://\:doi\:.org/\:10.1145/\:3569951.3597582
DOI
.UE .
.SH CONTACT INFORMATION
.PP
For patches, bug reports, or feature requests, please open an issue on
.UR https://\:github\:.com/\:upa/\:mscp
GitHub
.UE .
.SH AUTHORS
Ryo Nakamura <upa@haeena.net>
================================================
FILE: doc/mscp.rst
================================================
====
MSCP
====
:Date: v0.2.4
NAME
====
mscp - copy files over multiple SSH connections
SYNOPSIS
========
**mscp** [**-46vqDpdNh**] [ **-n** *NR_CONNECTIONS* ] [ **-m**
*COREMASK* ] [ **-u** *MAX_STARTUPS* ] [ **-I** *INTERVAL* ] [ **-W**
*CHECKPOINT* ] [ **-R** *CHECKPOINT* ] [ **-s** *MIN_CHUNK_SIZE* ] [
**-S** *MAX_CHUNK_SIZE* ] [ **-a** *NR_AHEAD* ] [ **-b** *BUF_SIZE* ] [
**-L** *LIMIT_BITRATE* ] [ **-l** *LOGIN_NAME* ] [ **-P** *PORT* ] [
**-F** *SSH_CONFIG* ] [ **-o** *SSH_OPTION* ] [ **-i** *IDENTITY* ] [
**-J** *DESTINATION* ] [ **-c** *CIPHER* ] [ **-M** *HMAC* ] [ **-C**
*COMPRESS* ] [ **-g** *CONGESTION* ] *source ... target*
DESCRIPTION
===========
**mscp** copies files over multiple SSH (SFTP) connections by multiple
threads. It enables transferring (1) multiple files simultaneously and
(2) a large file in parallel, reducing the transfer time for a lot
of/large files over networks.
The usage of **mscp** follows the **scp** command of *OpenSSH,* for
example:
::
$ mscp srcfile user@example.com:dstfile
Remote hosts only need to run standard **sshd** supporting the SFTP
subsystem, and users need to be able to **ssh** to the hosts as usual.
**mscp** does not require anything else.
**mscp** uses `libssh <https://www.libssh.org>`__ as its SSH
implementation. Thus, supported SSH features, for example,
authentication, encryption, and various options in ssh_config, follow
what *libssh* supports.
OPTIONS
=======
**-n NR_CONNECTIONS**
Specifies the number of SSH connections. The default value is
calculated from the number of CPU cores on the host with the
following formula: floor(log(nr_cores)*2)+1.
**-m COREMASK**
Configures CPU cores to be used by the hexadecimal bitmask. For
example, -m 0x25 pins threads onto CPU cores 0, 2, and 5. The default
value is not specified: all CPU cores are used and no threads are
pinned to any cores.
**-u MAX_STARTUPS**
Specifies the number of concurrent unauthenticated SSH connection
attempts. **sshd** limits the number of simultaneous SSH connection
attempts by *MaxStartups* in *sshd_config.* The default *MaxStartups*
is 10; thus, we set the default MAX_STARTUPS 8.
**-I INTERVAL**
Specifies the interval (in seconds) between SSH connection attempts.
Some firewall products treat SSH connection attempts from a single
source IP address for a short period as a brute force attack. This
option inserts intervals between the attempts to avoid being
determined as an attack. The default value is 0.
**-W CHECKPOINT**
Specifies a checkpoint file to save the state of a failed transfer.
When transferring fails due to, for example, connection disruption or
user interrupt, **mscp** writes the information about the remaining
files and chunks to the specified checkpoint file. **-W** option with
**-D** (dry-run mode) only writes a checkpoint file and exits.
**-R CHECKPOINT**
Specifies a checkpoint file to resume a transfer. When a checkpoint
file is passed, **mscp** reads the checkpoint to load a remote host,
copy direction, and files and their chunks to be transferred. Namely,
**mscp** can resume a past failed transfer from the checkpoint.
Resuming with a checkpoint does not require *source ... target*
arguments. Other SSH connection options, such as port number and
config file, should be specified as with the failed run. In addition,
checkpoint files have file paths as relative paths. Thus, you must
run **mscp** in the same working directory as the failed run. You can
see the contents of a checkpoint file with the **mscp -vv -D -R
CHECKPOINT** command (Dry-run mode). Note that the checkpoint file is
not automatically removed after the resumed transfer ends
successfully. Users should check the return value of **mscp** and
remove the checkpoint if it returns 0.
**-s MIN_CHUNK_SIZE**
Specifies the minimum chunk size. **mscp** divides a single file into
chunks and copies the chunks in parallel. The default value is 16M
bytes.
**-S MAX_CHUNK_SIZE**
Specifies the maximum chunk size. The default is file size divided by
the number of connections and devided by 4. If the calculated value
is smarller than the **MIN_CHUNK_SIZE** value, MIN_CHUNK_SIZE is
used.
**-a NR_AHEAD**
Specifies the number of inflight SFTP commands. The default value is
32.
**-b BUF_SIZE**
Specifies the buffer size for I/O and transfer over SFTP. The default
value is 16384. Note that the SSH specification restricts buffer size
delivered over SSH. Changing this value is not recommended at
present.
**-L LIMIT_BITRATE**
Limits the bitrate, specified with k (K), m (M), and g (G), e.g.,
100m indicates 100 Mbps.
**-4**
Uses IPv4 addresses only.
**-6**
Uses IPv6 addresses only.
**-v**
Increments the verbose output level.
**-q**
Quiet mode: turns off all outputs.
**-D**
Dry-run mode: it scans source files to be copied, calculates chunks,
resolves destination file paths, and exits. Dry-run mode with **-vv**
option can confirm files to be copied and their destination paths.
**-r**
No effect. **mscp** copies recursively if a source path is a
directory. This option exists for just compatibility.
**-l LOGIN_NAME**
Specifies the username to log in on the remote machine as with
*ssh(1).*
**-P PORT**
Specifies the port number to connect to on the remote machine as with
*scp(1).*
**-F SSH_CONFIG**
Specifies an alternative per-user ssh configuration file. Note that
acceptable options in the configuration file are what *libssh*
supports.
**-o SSH_OPTION**
Specifies ssh options in the format used in ssh_config. Note that
acceptable options are what *libssh* supports.
**-i IDENTITY**
Specifies the identity file for public key authentication.
**-J DESTINATION**
A shortcut to define a **ProxyJump** configuration directive. Each
SFTP session of **mscp** connects to the target host by first making
an **ssh** connection to the jump host described by *destination.*
**-c CIPHER**
Selects the cipher to use for encrypting the data transfer. See
**mscp -h** or **Ciphers** in `libssh
features <https://www.libssh.org/features/>`__.
**-M HMAC**
Specifies MAC hash algorithms. See **mscp -h** or **MAC hashes** in
`libssh features <https://www.libssh.org/features/>`__.
**-C COMPRESS**
Enables compression: yes, no, zlib, zlib@openssh.com. The default is
none. See `libssh features <https://www.libssh.org/features/>`__.
**-g CONGESTION**
Specifies the TCP congestion control algorithm to use (Linux only).
See **sysctl net.ipv4.tcp_allowed_congestion_control** for available
values.
**-p**
Preserves modification times and access times (file mode bits are
preserved by default).
**-d**
Increments the ssh debug output level.
**-N**
Enables Nagle's algorithm. It is disabled by default.
**-h**
Prints help.
EXIT STATUS
===========
Exit status is 0 on success, and >0 if an error occurs.
ENVIRONMENT
===========
**mscp** recognizes the following environment variables.
**MSCP_SSH_AUTH_PASSWORD**
This environment variable passes a password for password
authentication to establish SSH connections.
**MSCP_SSH_AUTH_PASSPHRASE**
This environment variable passes a passphrase for public-key
authentication for establishing SSH connections.
NOTES
=====
**mscp** uses glob(3) for globbing pathnames, including matching
patterns for local and remote paths. However, globbing on the *remote*
side does not work with musl libc (used in Alpine Linux and the
single-binary version of mscp) because musl libc does not support
GLOB_ALTDIRFUNC.
**mscp** does not support remote-to-remote copy, which **scp** supports.
EXAMPLES
========
Copy a local file to a remote host with different name:
::
$ mscp ~/src-file 10.0.0.1:copied-file
Copy a local file and a directory to /tmp at a remote host:
::
$ mscp ~/src-file dir1 10.0.0.1:/tmp
Save a checkpoint if transfer fails:
::
$ mscp -W mscp.checkpoint many-large-files 10.0.0.1:dst/
Check the remaining files and chunks, and resume the failed transfer:
::
# Dump the content of a checkpoint and exit (dry-run mode)
$ mscp -vv -D -R mscp.checkpoint
# resume transferring from the checkpoint
$ mscp -R mscp.checkpoint
In a long fat network, following options might improve performance:
::
$ mscp -n 64 -m 0xffff -a 64 -c aes128-gcm@openssh.com src 10.0.0.1:
**-n** increases the number of SSH connections than default, **-m** pins
threads to specific CPU cores, **-a** increases asynchronous inflight
SFTP WRITE/READ commands, and **-c aes128-gcm@openssh.com** will be
faster than the default chacha20-poly1305 cipher, particularly on hosts
that support AES-NI.
SEE ALSO
========
**scp**\ (1), **ssh**\ (1), **sshd**\ (8).
PAPER REFERENCE
===============
Ryo Nakamura and Yohei Kuga. 2023. Multi-threaded scp: Easy and Fast
File Transfer over SSH. In Practice and Experience in Advanced Research
Computing (PEARC '23). Association for Computing Machinery, New York,
NY, USA, 320–323. `DOI <https://doi.org/10.1145/3569951.3597582>`__.
CONTACT INFORMATION
===================
For patches, bug reports, or feature requests, please open an issue on
`GitHub <https://github.com/upa/mscp>`__.
AUTHORS
=======
Ryo Nakamura <upa@haeena.net>
================================================
FILE: include/config.h.in
================================================
/* SPDX-License-Identifier: GPL-3.0-only */
#ifndef _CONFIG_H_
#define _CONFIG_H_
#define MSCP_VERSION "@MSCP_VERSION@"
#define MSCP_BUILD_VERSION "@MSCP_BUILD_VERSION@"
/* Define to 1 if you have the strlcat function. */
#cmakedefine HAVE_STRLCAT 1
/* Define to 1 if you have the htonll function. */
#cmakedefine HAVE_HTONLL 1
/* Define to 1 if you have the ntohll function. */
#cmakedefine HAVE_NTOHLL 1
#endif /* _CONFIG_H_ */
================================================
FILE: include/mscp.h
================================================
/* SPDX-License-Identifier: GPL-3.0-only */
#ifndef _MSCP_H_
#define _MSCP_H_
/**
* @file mscp.h
*
* @brief mscp library header file.
*
* @mainpage
*
* libmscp is a library for multi-threaded scp. Project page is
* https://github.com/upa/mscp.
*
* All public APIs of libmscp are defined in mscp.h. Basic usage of
* libmscp is follows:
*
* 1. create mscp instance with mscp_init()
* 2. set remote host and copy direction with mscp_set_remote()
* 3. connect to remote host with mscp_connect()
* 4. add path to source files with mscp_add_src_path()
* 5. set path to destination with mscp_set_dst_path()
* 6. start to scan source files with mscp_scan()
* 7. start copy with mscp_start()
* 8. wait for copy finished with mscp_join()
* 9. cleanup mscp instance with mscp_cleanup() and mscp_free()
*/
#include <stdbool.h>
#include <limits.h>
#include <stdlib.h>
#define MSCP_DIRECTION_L2R 1 /** Indicates local to remote copy */
#define MSCP_DIRECTION_R2L 2 /** Indicates remote to local copy */
/**
* @struct mscp_opts
* @brief Structure configuring mscp.
*/
struct mscp_opts {
int nr_threads; /** number of copy threads */
int nr_ahead; /** number of SFTP commands on-the-fly */
size_t min_chunk_sz; /** minimum chunk size (default 64MB) */
size_t max_chunk_sz; /** maximum chunk size (default file size/nr_threads) */
size_t buf_sz; /** buffer size, default 16k. */
size_t bitrate; /** bits-per-seconds to limit bandwidth */
char *coremask; /** hex to specifiy usable cpu cores */
int max_startups; /** sshd MaxStartups concurrent connections */
int interval; /** interval between SSH connection attempts */
bool preserve_ts; /** preserve file timestamps */
int severity; /** messaging severity. set MSCP_SERVERITY_* */
};
/**
* @struct mscp_ssh_opts
* @brief Structure configuring SSH connections
*/
struct mscp_ssh_opts {
/* ssh options */
char *login_name; /** ssh username */
char *port; /** ssh port */
int ai_family; /** address family */
char *config; /** path to ssh_config, default ~/.ssh/config*/
char **options; /** array of ssh_config options, terminated by NULL */
char *identity; /** path to private key */
char *proxyjump; /** ProxyJump configuration directive (shortcut) */
char *cipher; /** cipher spec */
char *hmac; /** hmacp spec */
char *compress; /** yes, no, zlib@openssh.com */
char *ccalgo; /** TCP cc algorithm */
char *password; /** password auth passowrd */
char *passphrase; /** passphrase for private key */
int debug_level; /** inclirement libssh debug output level */
bool enable_nagle; /** enable Nagle's algorithm if true */
};
/** @def
* Environment variable that passes password for ssh password auth
*/
#define ENV_SSH_AUTH_PASSWORD "MSCP_SSH_AUTH_PASSWORD"
/** @def
* Environment vraible that passes passphrase for private key
*/
#define ENV_SSH_AUTH_PASSPHRASE "MSCP_SSH_AUTH_PASSPHRASE"
/**
* @struct mscp_stats
* @brief Structure to get mscp statistics
*/
struct mscp_stats {
size_t total; /** total bytes to be transferred */
size_t done; /** total bytes transferred */
};
/** Structure representing mscp instance */
struct mscp;
/**
* @brief Creates a new mscp instance.
*
* @param o options for configuring mscp.
* @param s options for configuring ssh connections.
*
* @retrun A new mscp instance or NULL on error.
*/
struct mscp *mscp_init(struct mscp_opts *o, struct mscp_ssh_opts *s);
/**
* @brief Set remote host and copy direction.
*
* @param remote_host remote host for file transer.
* @param direction copy direction, `MSCP_DIRECTION_L2R` or `MSCP_DIRECTION_R2L`
*
* @return 0 on success, < 0 if an error occured.
*/
int mscp_set_remote(struct mscp *m, const char *remote_host, int direction);
/**
* @brief Connect the first SSH connection. mscp_connect connects to
* remote host and initialize a SFTP session over the
* connection. mscp_scan() and mscp_start() require mscp_connect()
* beforehand.
*
* @param m mscp instance.
*
* @return 0 on success, < 0 if an error occured.
*/
int mscp_connect(struct mscp *m);
/* add a source file path to be copied */
/**
* @brief Add a source file path to be copied. The path indicates
* either a file or directory.
*
* @param m mscp instance.
* @param src_path source file path to be copied.
*
* @return 0 on success, < 0 if an error occured.
*/
int mscp_add_src_path(struct mscp *m, const char *src_path);
/**
* @brief Set the destination file path. The path indicates either a
* file, directory, or nonexistent path.
*
* @param m mscp instance.
* @param dst_path destination path to which source files copied.
*
* @return 0 on success, < 0 if an error occured.
*/
int mscp_set_dst_path(struct mscp *m, const char *dst_path);
/* scan source files, resolve destination file paths for all source
* files, and calculate chunks for all files. */
/**
* @brief Scan source paths and prepare. This function checks all
* source files (recursively), resolve paths on the destination side,
* and calculate file chunks. This function is non-blocking.
*
* @param m mscp instance.
*
* @return 0 on success, < 0 if an error occured.
*/
int mscp_scan(struct mscp *m);
/**
* @brief Join scan thread invoked by mscp_scan() if it
* runs. mscp_join() involves mscp_can_join(). Thus, there is no need
* to call this function alone.
*
* @param m mscp instance.
* @return 0 on success, < 0 if an error occured.
*/
int mscp_scan_join(struct mscp *m);
/**
* @brief get information about remote host and copy direction from a
* checkpoint file specified by *pathname. This functions returns
* remote host name to *renote, and the copy direction into *dir.
* Thus, you can call mscp_init with those values.
*
* @param pathname path to a checkpoint file.
* @param remote char buffer to which remote hostname is stored.
* @param len length of *remote.
* @param dir int to which the copy direction is stored.
*/
int mscp_checkpoint_get_remote(const char *pathname, char *remote, size_t len, int *dir);
/**
* @brief load information about untransferred files and chunks at the
* last transfer . mscp_checkpoint_load() loads files and associated
* chunks from the checkpoint file pointed by pathname. If you call
* mscp_checkpoint_load(), do not call mscp_scan().
*
* @param m mscp instance.
* @param pathname path to a checkpoint file.
* @return 0 on success, < 0 if an error occured.
*/
int mscp_checkpoint_load(struct mscp *m, const char *pathname);
/**
* @brief save information about untransferred files and chunks to a
* checkpoint file.
*
* @param m mscp instance.
* @param pathname path to a checkpoint file.
* @return 0 on success, < 0 if an error occured.
*/
int mscp_checkpoint_save(struct mscp *m, const char *pathname);
/**
* @brief Start to copy files. mscp_start() returns immediately. You
* can get statistics via mscp_get_stats() or messages via pipe set by
* mscp_opts.msg_fd or mscp_set_msg_fd(). mscp_stop() cancels mscp
* copy threads, and mscp_join() joins the threads.
*
* @param m mscp instance.
*
* @return number of threads on success, < 0 if an error occured.
*
* @see mscp_join()
*/
int mscp_start(struct mscp *m);
/**
* @brief Stop coping files.
*
* @param m mscp instance.
*/
void mscp_stop(struct mscp *m);
/**
* @brief Join copy threads. This function is blocking until all copy
* have done.
*
* @param m mscp instance.
*
* @return 0 on success, < 0 if an error occured.
*/
int mscp_join(struct mscp *m);
/**
* @brief Get statistics of copy.
*
* @param m mscp instance.
* @param s[out] statistics.
*/
void mscp_get_stats(struct mscp *m, struct mscp_stats *s);
/**
* @brief Cleanup the mscp instance. Before calling mscp_cleanup(),
* must call mscp_join(). After mscp_cleanup() called, the mscp
* instance can restart from mscp_connect(). Note that do not call
* mscp_cleanup() before callign mscp_join(). It causes crash (ToDo:
* check status of copy threads and return error when they are
* running).
*
* @param m mscp instance.
*/
void mscp_cleanup(struct mscp *m);
/**
* @brief Release the mscp instance. Note that do not call *
mscp_free() before calling mscp_join(). It causes crash (ToDo: check
* status of copy threads and return error when they are running).
*
* @param m mscp instance.
*/
void mscp_free(struct mscp *m);
/* messaging with mscp */
/**
* @enum mscp_serverity
* @brief Filter messages from libmscp with severity level.
*/
enum {
MSCP_SEVERITY_NONE = -1,
MSCP_SEVERITY_ERR = 0,
MSCP_SEVERITY_WARN = 1,
MSCP_SEVERITY_NOTICE = 2,
MSCP_SEVERITY_INFO = 3,
MSCP_SEVERITY_DEBUG = 4,
};
/**
* @brief Return available ciphers.
*/
const char **mscp_ssh_ciphers(void);
/**
* @brief Return available hmacs.
*/
const char **mscp_ssh_hmacs(void);
#endif /* _MSCP_H_ */
================================================
FILE: patch/README.md
================================================
Patches in this directory introduce enhancements for libssh including
`sftp_async_write()` and `sftp_async_write_end()`, derived from
https://github.com/limes-datentechnik-gmbh/libssh. See [Re: SFTP Write
async](https://archive.libssh.org/libssh/2020-06/0000004.html).
================================================
FILE: patch/libssh-0.10.4.patch
================================================
diff --git a/ConfigureChecks.cmake b/ConfigureChecks.cmake
index 7103f303..c64eb39d 100644
--- a/ConfigureChecks.cmake
+++ b/ConfigureChecks.cmake
@@ -258,6 +258,7 @@ if (UNIX)
check_library_exists(util forkpty "" HAVE_LIBUTIL)
check_function_exists(cfmakeraw HAVE_CFMAKERAW)
check_function_exists(__strtoull HAVE___STRTOULL)
+ check_symbol_exists(TCP_CONGESTION "netinet/tcp.h" HAVE_TCP_CONGESTION)
endif (UNIX)
set(LIBSSH_REQUIRED_LIBRARIES ${_REQUIRED_LIBRARIES} CACHE INTERNAL "libssh required system libraries")
diff --git a/config.h.cmake b/config.h.cmake
index 1357615b..1e915ead 100644
--- a/config.h.cmake
+++ b/config.h.cmake
@@ -237,6 +237,8 @@
#cmakedefine HAVE_GCC_BOUNDED_ATTRIBUTE 1
+#cmakedefine HAVE_TCP_CONGESTION 1
+
/* Define to 1 if you want to enable GSSAPI */
#cmakedefine WITH_GSSAPI 1
diff --git a/include/libssh/buffer.h b/include/libssh/buffer.h
index a55a1b40..e34e075c 100644
--- a/include/libssh/buffer.h
+++ b/include/libssh/buffer.h
@@ -33,6 +33,8 @@ int ssh_buffer_add_u8(ssh_buffer buffer, uint8_t data);
int ssh_buffer_add_u16(ssh_buffer buffer, uint16_t data);
int ssh_buffer_add_u32(ssh_buffer buffer, uint32_t data);
int ssh_buffer_add_u64(ssh_buffer buffer, uint64_t data);
+ssize_t ssh_buffer_add_func(ssh_buffer buffer, ssh_add_func f, size_t max_bytes,
+ void *userdata);
int ssh_buffer_validate_length(struct ssh_buffer_struct *buffer, size_t len);
diff --git a/include/libssh/libssh.h b/include/libssh/libssh.h
index 7857a77b..6b4d481c 100644
--- a/include/libssh/libssh.h
+++ b/include/libssh/libssh.h
@@ -402,6 +402,7 @@ enum ssh_options_e {
SSH_OPTIONS_GSSAPI_AUTH,
SSH_OPTIONS_GLOBAL_KNOWNHOSTS,
SSH_OPTIONS_NODELAY,
+ SSH_OPTIONS_CCALGO,
SSH_OPTIONS_PUBLICKEY_ACCEPTED_TYPES,
SSH_OPTIONS_PROCESS_CONFIG,
SSH_OPTIONS_REKEY_DATA,
@@ -833,6 +834,7 @@ LIBSSH_API const char* ssh_get_hmac_in(ssh_session session);
LIBSSH_API const char* ssh_get_hmac_out(ssh_session session);
LIBSSH_API ssh_buffer ssh_buffer_new(void);
+LIBSSH_API ssh_buffer ssh_buffer_new_size(uint32_t size, uint32_t headroom);
LIBSSH_API void ssh_buffer_free(ssh_buffer buffer);
#define SSH_BUFFER_FREE(x) \
do { if ((x) != NULL) { ssh_buffer_free(x); x = NULL; } } while(0)
@@ -843,6 +845,8 @@ LIBSSH_API void *ssh_buffer_get(ssh_buffer buffer);
LIBSSH_API uint32_t ssh_buffer_get_len(ssh_buffer buffer);
LIBSSH_API int ssh_session_set_disconnect_message(ssh_session session, const char *message);
+typedef ssize_t (*ssh_add_func) (void *ptr, size_t max_bytes, void *userdata);
+
#ifndef LIBSSH_LEGACY_0_4
#include "libssh/legacy.h"
#endif
diff --git a/include/libssh/session.h b/include/libssh/session.h
index d3e5787c..15183d1b 100644
--- a/include/libssh/session.h
+++ b/include/libssh/session.h
@@ -232,6 +232,7 @@ struct ssh_session_struct {
int gss_delegate_creds;
int flags;
int nodelay;
+ char *ccalgo;
bool config_processed;
uint8_t options_seen[SOC_MAX];
uint64_t rekey_data;
diff --git a/include/libssh/sftp.h b/include/libssh/sftp.h
index c855df8a..0fcdb9b8 100644
--- a/include/libssh/sftp.h
+++ b/include/libssh/sftp.h
@@ -565,6 +565,10 @@ LIBSSH_API int sftp_async_read(sftp_file file, void *data, uint32_t len, uint32_
*/
LIBSSH_API ssize_t sftp_write(sftp_file file, const void *buf, size_t count);
+LIBSSH_API ssize_t sftp_async_write(sftp_file file, ssh_add_func f, size_t count,
+ void *userdata, uint32_t* id);
+LIBSSH_API int sftp_async_write_end(sftp_file file, uint32_t id, int blocking);
+
/**
* @brief Seek to a specific location in a file.
*
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index c090fef7..e2f86309 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -435,6 +435,11 @@ if (BUILD_STATIC_LIB)
if (WIN32)
target_compile_definitions(ssh-static PUBLIC "LIBSSH_STATIC")
endif (WIN32)
+
+ install(TARGETS ssh-static
+ EXPORT libssh-config
+ LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
+ COMPONENT libraries)
endif (BUILD_STATIC_LIB)
message(STATUS "Threads_FOUND=${Threads_FOUND}")
diff --git a/src/buffer.c b/src/buffer.c
index e0068015..cc0caf35 100644
--- a/src/buffer.c
+++ b/src/buffer.c
@@ -141,6 +141,40 @@ struct ssh_buffer_struct *ssh_buffer_new(void)
return buf;
}
+/**
+ * @brief Create a new SSH buffer with a specified size and headroom.
+ *
+ * @param[in] len length for newly initialized SSH buffer.
+ * @param[in] headroom length for headroom
+ * @return A newly initialized SSH buffer, NULL on error.
+ */
+struct ssh_buffer_struct *ssh_buffer_new_size(uint32_t len, uint32_t headroom)
+{
+ struct ssh_buffer_struct *buf = NULL;
+ int rc;
+
+ if (len < headroom)
+ return NULL;
+
+ buf = calloc(1, sizeof(struct ssh_buffer_struct));
+ if (buf == NULL) {
+ return NULL;
+ }
+
+ rc = ssh_buffer_allocate_size(buf, len);
+ if (rc != 0) {
+ SAFE_FREE(buf);
+ return NULL;
+ }
+
+ buf->pos += headroom;
+ buf->used += headroom;
+
+ buffer_verify(buf);
+
+ return buf;
+}
+
/**
* @brief Deallocate a SSH buffer.
*
@@ -328,6 +362,49 @@ int ssh_buffer_add_data(struct ssh_buffer_struct *buffer, const void *data, uint
return 0;
}
+/**
+ * @brief Add data at the tail of a buffer by an external function
+ *
+ * @param[in] buffer The buffer to add data.
+ *
+ * @param[in] f function that adds data to the buffer.
+ *
+ * @param[in] max_bytes The maximum length of the data to add.
+ *
+ * @return actual bytes added on success, < 0 on error.
+ */
+ssize_t ssh_buffer_add_func(struct ssh_buffer_struct *buffer, ssh_add_func f,
+ size_t max_bytes, void *userdata)
+{
+ ssize_t actual;
+
+ if (buffer == NULL) {
+ return -1;
+ }
+
+ buffer_verify(buffer);
+
+ if (buffer->used + max_bytes < max_bytes) {
+ return -1;
+ }
+
+ if (buffer->allocated < (buffer->used + max_bytes)) {
+ if (buffer->pos > 0) {
+ buffer_shift(buffer);
+ }
+ if (realloc_buffer(buffer, buffer->used + max_bytes) < 0) {
+ return -1;
+ }
+ }
+
+ if ((actual = f(buffer->data + buffer->used, max_bytes, userdata)) < 0)
+ return -1;
+
+ buffer->used += actual;
+ buffer_verify(buffer);
+ return actual;
+}
+
/**
* @brief Ensure the buffer has at least a certain preallocated size.
*
diff --git a/src/connect.c b/src/connect.c
index 57e37e63..c02397d5 100644
--- a/src/connect.c
+++ b/src/connect.c
@@ -156,6 +156,20 @@ static int set_tcp_nodelay(socket_t socket)
sizeof(opt));
}
+static int set_tcp_ccalgo(socket_t socket, const char *ccalgo)
+{
+#ifdef HAVE_TCP_CONGESTION
+ return setsockopt(socket,
+ IPPROTO_TCP,
+ TCP_CONGESTION,
+ (void *)ccalgo,
+ strlen(ccalgo));
+#else
+ errno = ENOTSUP;
+ return -1;
+#endif
+}
+
/**
* @internal
*
@@ -256,6 +270,18 @@ socket_t ssh_connect_host_nonblocking(ssh_session session, const char *host,
}
}
+ if (session->opts.ccalgo) {
+ rc = set_tcp_ccalgo(s, session->opts.ccalgo);
+ if (rc < 0) {
+ ssh_set_error(session, SSH_FATAL,
+ "Failed to set TCP_CONGESTION on socket: %s",
+ ssh_strerror(errno, err_msg, SSH_ERRNO_MSG_MAX));
+ ssh_connect_socket_close(s);
+ s = -1;
+ continue;
+ }
+ }
+
errno = 0;
rc = connect(s, itr->ai_addr, itr->ai_addrlen);
if (rc == -1 && (errno != 0) && (errno != EINPROGRESS)) {
diff --git a/src/options.c b/src/options.c
index 49aaefa2..9f7360c3 100644
--- a/src/options.c
+++ b/src/options.c
@@ -210,6 +210,7 @@ int ssh_options_copy(ssh_session src, ssh_session *dest)
new->opts.gss_delegate_creds = src->opts.gss_delegate_creds;
new->opts.flags = src->opts.flags;
new->opts.nodelay = src->opts.nodelay;
+ new->opts.ccalgo = src->opts.ccalgo;
new->opts.config_processed = src->opts.config_processed;
new->common.log_verbosity = src->common.log_verbosity;
new->common.callbacks = src->common.callbacks;
@@ -450,6 +451,10 @@ int ssh_options_set_algo(ssh_session session,
* Set it to disable Nagle's Algorithm (TCP_NODELAY) on the
* session socket. (int, 0=false)
*
+ * - SSH_OPTIONS_CCALGO
+ * Set it to specify TCP congestion control algorithm on the
+ * session socket (Linux only). (int, 0=false)
+ *
* - SSH_OPTIONS_PROCESS_CONFIG
* Set it to false to disable automatic processing of per-user
* and system-wide OpenSSH configuration files. LibSSH
@@ -1013,6 +1018,20 @@ int ssh_options_set(ssh_session session, enum ssh_options_e type,
session->opts.nodelay = (*x & 0xff) > 0 ? 1 : 0;
}
break;
+ case SSH_OPTIONS_CCALGO:
+ v = value;
+ if (v == NULL || v[0] == '\0') {
+ ssh_set_error_invalid(session);
+ return -1;
+ } else {
+ SAFE_FREE(session->opts.ccalgo);
+ session->opts.ccalgo = strdup(v);
+ if (session->opts.ccalgo == NULL) {
+ ssh_set_error_oom(session);
+ return -1;
+ }
+ }
+ break;
case SSH_OPTIONS_PROCESS_CONFIG:
if (value == NULL) {
ssh_set_error_invalid(session);
diff --git a/src/session.c b/src/session.c
index 6025c133..6b197526 100644
--- a/src/session.c
+++ b/src/session.c
@@ -108,6 +108,7 @@ ssh_session ssh_new(void)
session->opts.fd = -1;
session->opts.compressionlevel = 7;
session->opts.nodelay = 0;
+ session->opts.ccalgo = NULL;
session->opts.flags = SSH_OPT_FLAG_PASSWORD_AUTH |
SSH_OPT_FLAG_PUBKEY_AUTH |
diff --git a/src/sftp.c b/src/sftp.c
index e01012a8..702623a0 100644
--- a/src/sftp.c
+++ b/src/sftp.c
@@ -2228,6 +2228,132 @@ ssize_t sftp_write(sftp_file file, const void *buf, size_t count) {
return -1; /* not reached */
}
+/*
+ * sftp_async_write is based on and sftp_async_write_end is copied from
+ * https://github.com/limes-datentechnik-gmbh/libssh
+ *
+ * sftp_async_write has some optimizations:
+ * - use ssh_buffer_new_size() to reduce realoc_buffer.
+ * - use ssh_buffer_add_func() to avoid memcpy from read buffer to ssh buffer.
+ */
+ssize_t sftp_async_write(sftp_file file, ssh_add_func f, size_t count, void *userdata,
+ uint32_t* id) {
+ sftp_session sftp = file->sftp;
+ ssh_buffer buffer;
+ uint32_t buf_sz;
+ ssize_t actual;
+ int len;
+ int packetlen;
+ int rc;
+
+#define HEADROOM 16
+ /* sftp_packet_write() prepends a 5-bytes (uint32_t length and
+ * 1-byte type) header to the head of the payload by
+ * ssh_buffer_prepend_data(). Inserting headroom by
+ * ssh_buffer_new_size() eliminates memcpy for prepending the
+ * header.
+ */
+
+ buf_sz = (HEADROOM + /* for header */
+ sizeof(uint32_t) + /* id */
+ ssh_string_len(file->handle) + 4 + /* file->handle */
+ sizeof(uint64_t) + /* file->offset */
+ sizeof(uint32_t) + /* count */
+ count); /* datastring */
+
+ buffer = ssh_buffer_new_size(buf_sz, HEADROOM);
+ if (buffer == NULL) {
+ ssh_set_error_oom(sftp->session);
+ return -1;
+ }
+
+ *id = sftp_get_new_id(file->sftp);
+
+ rc = ssh_buffer_pack(buffer,
+ "dSqd",
+ *id,
+ file->handle,
+ file->offset,
+ count); /* len of datastring */
+
+ if (rc != SSH_OK){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+
+ actual = ssh_buffer_add_func(buffer, f, count, userdata);
+ if (actual < 0){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+
+ packetlen=ssh_buffer_get_len(buffer)+5;
+ len = sftp_packet_write(file->sftp, SSH_FXP_WRITE, buffer);
+ ssh_buffer_free(buffer);
+ if (len < 0) {
+ return SSH_ERROR;
+ } else if (len != packetlen) {
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Could only send %d of %d bytes to remote host!", len, packetlen);
+ SSH_LOG(SSH_LOG_PACKET,
+ "Could not write as much data as expected");
+ return SSH_ERROR;
+ }
+
+ file->offset += actual;
+
+ return actual;
+}
+
+int sftp_async_write_end(sftp_file file, uint32_t id, int blocking) {
+ sftp_session sftp = file->sftp;
+ sftp_message msg = NULL;
+ sftp_status_message status;
+
+ msg = sftp_dequeue(sftp, id);
+ while (msg == NULL) {
+ if (!blocking && ssh_channel_poll(sftp->channel, 0) == 0) {
+ /* we cannot block */
+ return SSH_AGAIN;
+ }
+ if (sftp_read_and_dispatch(sftp) < 0) {
+ /* something nasty has happened */
+ return SSH_ERROR;
+ }
+ msg = sftp_dequeue(sftp, id);
+ }
+
+ switch (msg->packet_type) {
+ case SSH_FXP_STATUS:
+ status = parse_status_msg(msg);
+ sftp_message_free(msg);
+ if (status == NULL) {
+ return SSH_ERROR;
+ }
+ sftp_set_error(sftp, status->status);
+ switch (status->status) {
+ case SSH_FX_OK:
+ status_msg_free(status);
+ return SSH_OK;
+ default:
+ break;
+ }
+ ssh_set_error(sftp->session, SSH_REQUEST_DENIED,
+ "SFTP server: %s", status->errormsg);
+ status_msg_free(status);
+ return SSH_ERROR;
+ default:
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Received message %d during write!", msg->packet_type);
+ sftp_message_free(msg);
+ return SSH_ERROR;
+ }
+
+ return SSH_ERROR; /* not reached */
+}
+
/* Seek to a specific location in a file. */
int sftp_seek(sftp_file file, uint32_t new_offset) {
if (file == NULL) {
================================================
FILE: patch/libssh-0.10.6-2-g6f1b1e76.patch
================================================
diff --git a/CMakeLists.txt b/CMakeLists.txt
index a64b7708..c6344a5a 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 3.3.0)
+cmake_minimum_required(VERSION 3.13.0)
cmake_policy(SET CMP0048 NEW)
# Specify search path for CMake modules to be loaded by include()
diff --git a/ConfigureChecks.cmake b/ConfigureChecks.cmake
index 9de10225..0f3d20ed 100644
--- a/ConfigureChecks.cmake
+++ b/ConfigureChecks.cmake
@@ -258,6 +258,7 @@ if (UNIX)
check_library_exists(util forkpty "" HAVE_LIBUTIL)
check_function_exists(cfmakeraw HAVE_CFMAKERAW)
check_function_exists(__strtoull HAVE___STRTOULL)
+ check_symbol_exists(TCP_CONGESTION "netinet/tcp.h" HAVE_TCP_CONGESTION)
endif (UNIX)
set(LIBSSH_REQUIRED_LIBRARIES ${_REQUIRED_LIBRARIES} CACHE INTERNAL "libssh required system libraries")
diff --git a/config.h.cmake b/config.h.cmake
index cc83734d..f74cd03b 100644
--- a/config.h.cmake
+++ b/config.h.cmake
@@ -237,6 +237,8 @@
#cmakedefine HAVE_GCC_BOUNDED_ATTRIBUTE 1
+#cmakedefine HAVE_TCP_CONGESTION 1
+
/* Define to 1 if you want to enable GSSAPI */
#cmakedefine WITH_GSSAPI 1
diff --git a/include/libssh/buffer.h b/include/libssh/buffer.h
index 1fce7b76..b64d1455 100644
--- a/include/libssh/buffer.h
+++ b/include/libssh/buffer.h
@@ -37,6 +37,8 @@ int ssh_buffer_add_u8(ssh_buffer buffer, uint8_t data);
int ssh_buffer_add_u16(ssh_buffer buffer, uint16_t data);
int ssh_buffer_add_u32(ssh_buffer buffer, uint32_t data);
int ssh_buffer_add_u64(ssh_buffer buffer, uint64_t data);
+ssize_t ssh_buffer_add_func(ssh_buffer buffer, ssh_add_func f, size_t max_bytes,
+ void *userdata);
int ssh_buffer_validate_length(struct ssh_buffer_struct *buffer, size_t len);
diff --git a/include/libssh/libssh.h b/include/libssh/libssh.h
index 669a0a96..26b20f3f 100644
--- a/include/libssh/libssh.h
+++ b/include/libssh/libssh.h
@@ -368,6 +368,7 @@ enum ssh_options_e {
SSH_OPTIONS_HOST,
SSH_OPTIONS_PORT,
SSH_OPTIONS_PORT_STR,
+ SSH_OPTIONS_AI_FAMILY,
SSH_OPTIONS_FD,
SSH_OPTIONS_USER,
SSH_OPTIONS_SSH_DIR,
@@ -402,6 +403,7 @@ enum ssh_options_e {
SSH_OPTIONS_GSSAPI_AUTH,
SSH_OPTIONS_GLOBAL_KNOWNHOSTS,
SSH_OPTIONS_NODELAY,
+ SSH_OPTIONS_CCALGO,
SSH_OPTIONS_PUBLICKEY_ACCEPTED_TYPES,
SSH_OPTIONS_PROCESS_CONFIG,
SSH_OPTIONS_REKEY_DATA,
@@ -833,6 +835,7 @@ LIBSSH_API const char* ssh_get_hmac_in(ssh_session session);
LIBSSH_API const char* ssh_get_hmac_out(ssh_session session);
LIBSSH_API ssh_buffer ssh_buffer_new(void);
+LIBSSH_API ssh_buffer ssh_buffer_new_size(uint32_t size, uint32_t headroom);
LIBSSH_API void ssh_buffer_free(ssh_buffer buffer);
#define SSH_BUFFER_FREE(x) \
do { if ((x) != NULL) { ssh_buffer_free(x); x = NULL; } } while(0)
@@ -843,6 +846,11 @@ LIBSSH_API void *ssh_buffer_get(ssh_buffer buffer);
LIBSSH_API uint32_t ssh_buffer_get_len(ssh_buffer buffer);
LIBSSH_API int ssh_session_set_disconnect_message(ssh_session session, const char *message);
+typedef ssize_t (*ssh_add_func) (void *ptr, size_t max_bytes, void *userdata);
+
+LIBSSH_API const char **ssh_ciphers(void);
+LIBSSH_API const char **ssh_hmacs(void);
+
#ifndef LIBSSH_LEGACY_0_4
#include "libssh/legacy.h"
#endif
diff --git a/include/libssh/session.h b/include/libssh/session.h
index 97936195..e4fc4fce 100644
--- a/include/libssh/session.h
+++ b/include/libssh/session.h
@@ -249,6 +249,7 @@ struct ssh_session_struct {
unsigned long timeout; /* seconds */
unsigned long timeout_usec;
uint16_t port;
+ int ai_family;
socket_t fd;
int StrictHostKeyChecking;
char compressionlevel;
@@ -258,6 +259,7 @@ struct ssh_session_struct {
int flags;
int exp_flags;
int nodelay;
+ char *ccalgo;
bool config_processed;
uint8_t options_seen[SOC_MAX];
uint64_t rekey_data;
diff --git a/include/libssh/sftp.h b/include/libssh/sftp.h
index c713466e..e27fe326 100644
--- a/include/libssh/sftp.h
+++ b/include/libssh/sftp.h
@@ -565,6 +565,10 @@ LIBSSH_API int sftp_async_read(sftp_file file, void *data, uint32_t len, uint32_
*/
LIBSSH_API ssize_t sftp_write(sftp_file file, const void *buf, size_t count);
+LIBSSH_API ssize_t sftp_async_write(sftp_file file, ssh_add_func f, size_t count,
+ void *userdata, uint32_t* id);
+LIBSSH_API int sftp_async_write_end(sftp_file file, uint32_t id, int blocking);
+
/**
* @brief Seek to a specific location in a file.
*
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 807313b5..86487087 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -448,6 +448,11 @@ if (BUILD_STATIC_LIB)
if (WIN32)
target_compile_definitions(ssh-static PUBLIC "LIBSSH_STATIC")
endif (WIN32)
+
+ install(TARGETS ssh-static
+ EXPORT libssh-config
+ LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
+ COMPONENT libraries)
endif (BUILD_STATIC_LIB)
message(STATUS "Threads_FOUND=${Threads_FOUND}")
diff --git a/src/buffer.c b/src/buffer.c
index 8991e006..e0414801 100644
--- a/src/buffer.c
+++ b/src/buffer.c
@@ -142,6 +142,40 @@ struct ssh_buffer_struct *ssh_buffer_new(void)
return buf;
}
+/**
+ * @brief Create a new SSH buffer with a specified size and headroom.
+ *
+ * @param[in] len length for newly initialized SSH buffer.
+ * @param[in] headroom length for headroom
+ * @return A newly initialized SSH buffer, NULL on error.
+ */
+struct ssh_buffer_struct *ssh_buffer_new_size(uint32_t len, uint32_t headroom)
+{
+ struct ssh_buffer_struct *buf = NULL;
+ int rc;
+
+ if (len < headroom)
+ return NULL;
+
+ buf = calloc(1, sizeof(struct ssh_buffer_struct));
+ if (buf == NULL) {
+ return NULL;
+ }
+
+ rc = ssh_buffer_allocate_size(buf, len);
+ if (rc != 0) {
+ SAFE_FREE(buf);
+ return NULL;
+ }
+
+ buf->pos += headroom;
+ buf->used += headroom;
+
+ buffer_verify(buf);
+
+ return buf;
+}
+
/**
* @brief Deallocate a SSH buffer.
*
@@ -329,6 +363,49 @@ int ssh_buffer_add_data(struct ssh_buffer_struct *buffer, const void *data, uint
return 0;
}
+/**
+ * @brief Add data at the tail of a buffer by an external function
+ *
+ * @param[in] buffer The buffer to add data.
+ *
+ * @param[in] f function that adds data to the buffer.
+ *
+ * @param[in] max_bytes The maximum length of the data to add.
+ *
+ * @return actual bytes added on success, < 0 on error.
+ */
+ssize_t ssh_buffer_add_func(struct ssh_buffer_struct *buffer, ssh_add_func f,
+ size_t max_bytes, void *userdata)
+{
+ ssize_t actual;
+
+ if (buffer == NULL) {
+ return -1;
+ }
+
+ buffer_verify(buffer);
+
+ if (buffer->used + max_bytes < max_bytes) {
+ return -1;
+ }
+
+ if (buffer->allocated < (buffer->used + max_bytes)) {
+ if (buffer->pos > 0) {
+ buffer_shift(buffer);
+ }
+ if (realloc_buffer(buffer, buffer->used + max_bytes) < 0) {
+ return -1;
+ }
+ }
+
+ if ((actual = f(buffer->data + buffer->used, max_bytes, userdata)) < 0)
+ return -1;
+
+ buffer->used += actual;
+ buffer_verify(buffer);
+ return actual;
+}
+
/**
* @brief Ensure the buffer has at least a certain preallocated size.
*
diff --git a/src/connect.c b/src/connect.c
index 15cae644..02ef43b4 100644
--- a/src/connect.c
+++ b/src/connect.c
@@ -114,7 +114,7 @@ static int ssh_connect_socket_close(socket_t s)
#endif
}
-static int getai(const char *host, int port, struct addrinfo **ai)
+static int getai(const char *host, int port, int ai_family, struct addrinfo **ai)
{
const char *service = NULL;
struct addrinfo hints;
@@ -123,7 +123,7 @@ static int getai(const char *host, int port, struct addrinfo **ai)
ZERO_STRUCT(hints);
hints.ai_protocol = IPPROTO_TCP;
- hints.ai_family = PF_UNSPEC;
+ hints.ai_family = ai_family > 0 ? ai_family : PF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
if (port == 0) {
@@ -156,6 +156,20 @@ static int set_tcp_nodelay(socket_t socket)
sizeof(opt));
}
+static int set_tcp_ccalgo(socket_t socket, const char *ccalgo)
+{
+#ifdef HAVE_TCP_CONGESTION
+ return setsockopt(socket,
+ IPPROTO_TCP,
+ TCP_CONGESTION,
+ (void *)ccalgo,
+ strlen(ccalgo));
+#else
+ errno = ENOTSUP;
+ return -1;
+#endif
+}
+
/**
* @internal
*
@@ -173,7 +187,7 @@ socket_t ssh_connect_host_nonblocking(ssh_session session, const char *host,
struct addrinfo *ai = NULL;
struct addrinfo *itr = NULL;
- rc = getai(host, port, &ai);
+ rc = getai(host, port, session->opts.ai_family, &ai);
if (rc != 0) {
ssh_set_error(session, SSH_FATAL,
"Failed to resolve hostname %s (%s)",
@@ -199,7 +213,7 @@ socket_t ssh_connect_host_nonblocking(ssh_session session, const char *host,
SSH_LOG(SSH_LOG_PACKET, "Resolving %s", bind_addr);
- rc = getai(bind_addr, 0, &bind_ai);
+ rc = getai(bind_addr, 0, session->opts.ai_family, &bind_ai);
if (rc != 0) {
ssh_set_error(session, SSH_FATAL,
"Failed to resolve bind address %s (%s)",
@@ -256,6 +270,18 @@ socket_t ssh_connect_host_nonblocking(ssh_session session, const char *host,
}
}
+ if (session->opts.ccalgo) {
+ rc = set_tcp_ccalgo(s, session->opts.ccalgo);
+ if (rc < 0) {
+ ssh_set_error(session, SSH_FATAL,
+ "Failed to set TCP_CONGESTION on socket: %s",
+ ssh_strerror(errno, err_msg, SSH_ERRNO_MSG_MAX));
+ ssh_connect_socket_close(s);
+ s = -1;
+ continue;
+ }
+ }
+
errno = 0;
rc = connect(s, itr->ai_addr, itr->ai_addrlen);
if (rc == -1 && (errno != 0) && (errno != EINPROGRESS)) {
diff --git a/src/misc.c b/src/misc.c
index 7081f12a..e3879fe4 100644
--- a/src/misc.c
+++ b/src/misc.c
@@ -71,6 +71,8 @@
#include "libssh/priv.h"
#include "libssh/misc.h"
#include "libssh/session.h"
+#include "libssh/wrapper.h"
+#include "libssh/crypto.h"
#ifdef HAVE_LIBGCRYPT
#define GCRYPT_STRING "/gnutls"
@@ -2074,4 +2076,40 @@ int ssh_check_hostname_syntax(const char *hostname)
return SSH_OK;
}
+/**
+ * @brief Return supported cipher names
+ * @return The list of cipher names.
+ */
+const char **ssh_ciphers(void)
+{
+ struct ssh_cipher_struct *tab=ssh_get_ciphertab();
+ static const char *ciphers[32];
+ int n;
+
+ memset(ciphers, 0, sizeof(*ciphers));
+
+ for (n = 0; tab[n].name != NULL; n++) {
+ ciphers[n] = tab[n].name;
+ }
+ return ciphers;
+}
+
+/**
+ * @brief Return supported hmac names
+ * @return The list of hmac names.
+ */
+const char **ssh_hmacs(void)
+{
+ struct ssh_hmac_struct *tab=ssh_get_hmactab();
+ static const char *hmacs[32];
+ int n;
+
+ memset(hmacs, 0, sizeof(*hmacs));
+
+ for (n = 0; tab[n].name != NULL; n++) {
+ hmacs[n] = tab[n].name;
+ }
+ return hmacs;
+}
+
/** @} */
diff --git a/src/options.c b/src/options.c
index b3ecffe1..8de24ed6 100644
--- a/src/options.c
+++ b/src/options.c
@@ -217,6 +217,7 @@ int ssh_options_copy(ssh_session src, ssh_session *dest)
new->opts.gss_delegate_creds = src->opts.gss_delegate_creds;
new->opts.flags = src->opts.flags;
new->opts.nodelay = src->opts.nodelay;
+ new->opts.ccalgo = src->opts.ccalgo;
new->opts.config_processed = src->opts.config_processed;
new->common.log_verbosity = src->common.log_verbosity;
new->common.callbacks = src->common.callbacks;
@@ -268,6 +269,9 @@ int ssh_options_set_algo(ssh_session session,
* - SSH_OPTIONS_PORT_STR:
* The port to connect to (const char *).
*
+ * - SSH_OPTIONS_AI_FAMILY:
+ * The address family for connecting (int *).
+ *
* - SSH_OPTIONS_FD:
* The file descriptor to use (socket_t).\n
* \n
@@ -458,6 +462,10 @@ int ssh_options_set_algo(ssh_session session,
* Set it to disable Nagle's Algorithm (TCP_NODELAY) on the
* session socket. (int, 0=false)
*
+ * - SSH_OPTIONS_CCALGO
+ * Set it to specify TCP congestion control algorithm on the
+ * session socket (Linux only). (int, 0=false)
+ *
* - SSH_OPTIONS_PROCESS_CONFIG
* Set it to false to disable automatic processing of per-user
* and system-wide OpenSSH configuration files. LibSSH
@@ -571,6 +579,21 @@ int ssh_options_set(ssh_session session, enum ssh_options_e type,
session->opts.port = i & 0xffffU;
}
break;
+ case SSH_OPTIONS_AI_FAMILY:
+ if (value == NULL) {
+ session->opts.ai_family = 0;
+ ssh_set_error_invalid(session);
+ return -1;
+ } else {
+ int *x = (int *) value;
+ if (*x < 0) {
+ session->opts.ai_family = 0;
+ ssh_set_error_invalid(session);
+ return -1;
+ }
+ session->opts.ai_family = *x;
+ }
+ break;
case SSH_OPTIONS_FD:
if (value == NULL) {
session->opts.fd = SSH_INVALID_SOCKET;
@@ -1017,6 +1040,20 @@ int ssh_options_set(ssh_session session, enum ssh_options_e type,
session->opts.nodelay = (*x & 0xff) > 0 ? 1 : 0;
}
break;
+ case SSH_OPTIONS_CCALGO:
+ v = value;
+ if (v == NULL || v[0] == '\0') {
+ ssh_set_error_invalid(session);
+ return -1;
+ } else {
+ SAFE_FREE(session->opts.ccalgo);
+ session->opts.ccalgo = strdup(v);
+ if (session->opts.ccalgo == NULL) {
+ ssh_set_error_oom(session);
+ return -1;
+ }
+ }
+ break;
case SSH_OPTIONS_PROCESS_CONFIG:
if (value == NULL) {
ssh_set_error_invalid(session);
diff --git a/src/session.c b/src/session.c
index 8c509699..307388e5 100644
--- a/src/session.c
+++ b/src/session.c
@@ -105,9 +105,11 @@ ssh_session ssh_new(void)
/* OPTIONS */
session->opts.StrictHostKeyChecking = 1;
session->opts.port = 22;
+ session->opts.ai_family = 0;
session->opts.fd = -1;
session->opts.compressionlevel = 7;
session->opts.nodelay = 0;
+ session->opts.ccalgo = NULL;
session->opts.flags = SSH_OPT_FLAG_PASSWORD_AUTH |
SSH_OPT_FLAG_PUBKEY_AUTH |
diff --git a/src/sftp.c b/src/sftp.c
index e01012a8..702623a0 100644
--- a/src/sftp.c
+++ b/src/sftp.c
@@ -2228,6 +2228,132 @@ ssize_t sftp_write(sftp_file file, const void *buf, size_t count) {
return -1; /* not reached */
}
+/*
+ * sftp_async_write is based on and sftp_async_write_end is copied from
+ * https://github.com/limes-datentechnik-gmbh/libssh
+ *
+ * sftp_async_write has some optimizations:
+ * - use ssh_buffer_new_size() to reduce realoc_buffer.
+ * - use ssh_buffer_add_func() to avoid memcpy from read buffer to ssh buffer.
+ */
+ssize_t sftp_async_write(sftp_file file, ssh_add_func f, size_t count, void *userdata,
+ uint32_t* id) {
+ sftp_session sftp = file->sftp;
+ ssh_buffer buffer;
+ uint32_t buf_sz;
+ ssize_t actual;
+ int len;
+ int packetlen;
+ int rc;
+
+#define HEADROOM 16
+ /* sftp_packet_write() prepends a 5-bytes (uint32_t length and
+ * 1-byte type) header to the head of the payload by
+ * ssh_buffer_prepend_data(). Inserting headroom by
+ * ssh_buffer_new_size() eliminates memcpy for prepending the
+ * header.
+ */
+
+ buf_sz = (HEADROOM + /* for header */
+ sizeof(uint32_t) + /* id */
+ ssh_string_len(file->handle) + 4 + /* file->handle */
+ sizeof(uint64_t) + /* file->offset */
+ sizeof(uint32_t) + /* count */
+ count); /* datastring */
+
+ buffer = ssh_buffer_new_size(buf_sz, HEADROOM);
+ if (buffer == NULL) {
+ ssh_set_error_oom(sftp->session);
+ return -1;
+ }
+
+ *id = sftp_get_new_id(file->sftp);
+
+ rc = ssh_buffer_pack(buffer,
+ "dSqd",
+ *id,
+ file->handle,
+ file->offset,
+ count); /* len of datastring */
+
+ if (rc != SSH_OK){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+
+ actual = ssh_buffer_add_func(buffer, f, count, userdata);
+ if (actual < 0){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+
+ packetlen=ssh_buffer_get_len(buffer)+5;
+ len = sftp_packet_write(file->sftp, SSH_FXP_WRITE, buffer);
+ ssh_buffer_free(buffer);
+ if (len < 0) {
+ return SSH_ERROR;
+ } else if (len != packetlen) {
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Could only send %d of %d bytes to remote host!", len, packetlen);
+ SSH_LOG(SSH_LOG_PACKET,
+ "Could not write as much data as expected");
+ return SSH_ERROR;
+ }
+
+ file->offset += actual;
+
+ return actual;
+}
+
+int sftp_async_write_end(sftp_file file, uint32_t id, int blocking) {
+ sftp_session sftp = file->sftp;
+ sftp_message msg = NULL;
+ sftp_status_message status;
+
+ msg = sftp_dequeue(sftp, id);
+ while (msg == NULL) {
+ if (!blocking && ssh_channel_poll(sftp->channel, 0) == 0) {
+ /* we cannot block */
+ return SSH_AGAIN;
+ }
+ if (sftp_read_and_dispatch(sftp) < 0) {
+ /* something nasty has happened */
+ return SSH_ERROR;
+ }
+ msg = sftp_dequeue(sftp, id);
+ }
+
+ switch (msg->packet_type) {
+ case SSH_FXP_STATUS:
+ status = parse_status_msg(msg);
+ sftp_message_free(msg);
+ if (status == NULL) {
+ return SSH_ERROR;
+ }
+ sftp_set_error(sftp, status->status);
+ switch (status->status) {
+ case SSH_FX_OK:
+ status_msg_free(status);
+ return SSH_OK;
+ default:
+ break;
+ }
+ ssh_set_error(sftp->session, SSH_REQUEST_DENIED,
+ "SFTP server: %s", status->errormsg);
+ status_msg_free(status);
+ return SSH_ERROR;
+ default:
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Received message %d during write!", msg->packet_type);
+ sftp_message_free(msg);
+ return SSH_ERROR;
+ }
+
+ return SSH_ERROR; /* not reached */
+}
+
/* Seek to a specific location in a file. */
int sftp_seek(sftp_file file, uint32_t new_offset) {
if (file == NULL) {
================================================
FILE: patch/libssh-0.10.6.patch
================================================
diff --git a/ConfigureChecks.cmake b/ConfigureChecks.cmake
index 9de10225..0f3d20ed 100644
--- a/ConfigureChecks.cmake
+++ b/ConfigureChecks.cmake
@@ -258,6 +258,7 @@ if (UNIX)
check_library_exists(util forkpty "" HAVE_LIBUTIL)
check_function_exists(cfmakeraw HAVE_CFMAKERAW)
check_function_exists(__strtoull HAVE___STRTOULL)
+ check_symbol_exists(TCP_CONGESTION "netinet/tcp.h" HAVE_TCP_CONGESTION)
endif (UNIX)
set(LIBSSH_REQUIRED_LIBRARIES ${_REQUIRED_LIBRARIES} CACHE INTERNAL "libssh required system libraries")
diff --git a/config.h.cmake b/config.h.cmake
index cc83734d..f74cd03b 100644
--- a/config.h.cmake
+++ b/config.h.cmake
@@ -237,6 +237,8 @@
#cmakedefine HAVE_GCC_BOUNDED_ATTRIBUTE 1
+#cmakedefine HAVE_TCP_CONGESTION 1
+
/* Define to 1 if you want to enable GSSAPI */
#cmakedefine WITH_GSSAPI 1
diff --git a/include/libssh/buffer.h b/include/libssh/buffer.h
index 1fce7b76..b64d1455 100644
--- a/include/libssh/buffer.h
+++ b/include/libssh/buffer.h
@@ -37,6 +37,8 @@ int ssh_buffer_add_u8(ssh_buffer buffer, uint8_t data);
int ssh_buffer_add_u16(ssh_buffer buffer, uint16_t data);
int ssh_buffer_add_u32(ssh_buffer buffer, uint32_t data);
int ssh_buffer_add_u64(ssh_buffer buffer, uint64_t data);
+ssize_t ssh_buffer_add_func(ssh_buffer buffer, ssh_add_func f, size_t max_bytes,
+ void *userdata);
int ssh_buffer_validate_length(struct ssh_buffer_struct *buffer, size_t len);
diff --git a/include/libssh/libssh.h b/include/libssh/libssh.h
index 669a0a96..b6a93ac7 100644
--- a/include/libssh/libssh.h
+++ b/include/libssh/libssh.h
@@ -402,6 +402,7 @@ enum ssh_options_e {
SSH_OPTIONS_GSSAPI_AUTH,
SSH_OPTIONS_GLOBAL_KNOWNHOSTS,
SSH_OPTIONS_NODELAY,
+ SSH_OPTIONS_CCALGO,
SSH_OPTIONS_PUBLICKEY_ACCEPTED_TYPES,
SSH_OPTIONS_PROCESS_CONFIG,
SSH_OPTIONS_REKEY_DATA,
@@ -833,6 +834,7 @@ LIBSSH_API const char* ssh_get_hmac_in(ssh_session session);
LIBSSH_API const char* ssh_get_hmac_out(ssh_session session);
LIBSSH_API ssh_buffer ssh_buffer_new(void);
+LIBSSH_API ssh_buffer ssh_buffer_new_size(uint32_t size, uint32_t headroom);
LIBSSH_API void ssh_buffer_free(ssh_buffer buffer);
#define SSH_BUFFER_FREE(x) \
do { if ((x) != NULL) { ssh_buffer_free(x); x = NULL; } } while(0)
@@ -843,6 +845,8 @@ LIBSSH_API void *ssh_buffer_get(ssh_buffer buffer);
LIBSSH_API uint32_t ssh_buffer_get_len(ssh_buffer buffer);
LIBSSH_API int ssh_session_set_disconnect_message(ssh_session session, const char *message);
+typedef ssize_t (*ssh_add_func) (void *ptr, size_t max_bytes, void *userdata);
+
#ifndef LIBSSH_LEGACY_0_4
#include "libssh/legacy.h"
#endif
diff --git a/include/libssh/session.h b/include/libssh/session.h
index 97936195..e4a7f80c 100644
--- a/include/libssh/session.h
+++ b/include/libssh/session.h
@@ -258,6 +258,7 @@ struct ssh_session_struct {
int flags;
int exp_flags;
int nodelay;
+ char *ccalgo;
bool config_processed;
uint8_t options_seen[SOC_MAX];
uint64_t rekey_data;
diff --git a/include/libssh/sftp.h b/include/libssh/sftp.h
index c713466e..e27fe326 100644
--- a/include/libssh/sftp.h
+++ b/include/libssh/sftp.h
@@ -565,6 +565,10 @@ LIBSSH_API int sftp_async_read(sftp_file file, void *data, uint32_t len, uint32_
*/
LIBSSH_API ssize_t sftp_write(sftp_file file, const void *buf, size_t count);
+LIBSSH_API ssize_t sftp_async_write(sftp_file file, ssh_add_func f, size_t count,
+ void *userdata, uint32_t* id);
+LIBSSH_API int sftp_async_write_end(sftp_file file, uint32_t id, int blocking);
+
/**
* @brief Seek to a specific location in a file.
*
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 807313b5..86487087 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -448,6 +448,11 @@ if (BUILD_STATIC_LIB)
if (WIN32)
target_compile_definitions(ssh-static PUBLIC "LIBSSH_STATIC")
endif (WIN32)
+
+ install(TARGETS ssh-static
+ EXPORT libssh-config
+ LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
+ COMPONENT libraries)
endif (BUILD_STATIC_LIB)
message(STATUS "Threads_FOUND=${Threads_FOUND}")
diff --git a/src/buffer.c b/src/buffer.c
index 8991e006..e0414801 100644
--- a/src/buffer.c
+++ b/src/buffer.c
@@ -142,6 +142,40 @@ struct ssh_buffer_struct *ssh_buffer_new(void)
return buf;
}
+/**
+ * @brief Create a new SSH buffer with a specified size and headroom.
+ *
+ * @param[in] len length for newly initialized SSH buffer.
+ * @param[in] headroom length for headroom
+ * @return A newly initialized SSH buffer, NULL on error.
+ */
+struct ssh_buffer_struct *ssh_buffer_new_size(uint32_t len, uint32_t headroom)
+{
+ struct ssh_buffer_struct *buf = NULL;
+ int rc;
+
+ if (len < headroom)
+ return NULL;
+
+ buf = calloc(1, sizeof(struct ssh_buffer_struct));
+ if (buf == NULL) {
+ return NULL;
+ }
+
+ rc = ssh_buffer_allocate_size(buf, len);
+ if (rc != 0) {
+ SAFE_FREE(buf);
+ return NULL;
+ }
+
+ buf->pos += headroom;
+ buf->used += headroom;
+
+ buffer_verify(buf);
+
+ return buf;
+}
+
/**
* @brief Deallocate a SSH buffer.
*
@@ -329,6 +363,49 @@ int ssh_buffer_add_data(struct ssh_buffer_struct *buffer, const void *data, uint
return 0;
}
+/**
+ * @brief Add data at the tail of a buffer by an external function
+ *
+ * @param[in] buffer The buffer to add data.
+ *
+ * @param[in] f function that adds data to the buffer.
+ *
+ * @param[in] max_bytes The maximum length of the data to add.
+ *
+ * @return actual bytes added on success, < 0 on error.
+ */
+ssize_t ssh_buffer_add_func(struct ssh_buffer_struct *buffer, ssh_add_func f,
+ size_t max_bytes, void *userdata)
+{
+ ssize_t actual;
+
+ if (buffer == NULL) {
+ return -1;
+ }
+
+ buffer_verify(buffer);
+
+ if (buffer->used + max_bytes < max_bytes) {
+ return -1;
+ }
+
+ if (buffer->allocated < (buffer->used + max_bytes)) {
+ if (buffer->pos > 0) {
+ buffer_shift(buffer);
+ }
+ if (realloc_buffer(buffer, buffer->used + max_bytes) < 0) {
+ return -1;
+ }
+ }
+
+ if ((actual = f(buffer->data + buffer->used, max_bytes, userdata)) < 0)
+ return -1;
+
+ buffer->used += actual;
+ buffer_verify(buffer);
+ return actual;
+}
+
/**
* @brief Ensure the buffer has at least a certain preallocated size.
*
diff --git a/src/connect.c b/src/connect.c
index 15cae644..e7520f40 100644
--- a/src/connect.c
+++ b/src/connect.c
@@ -156,6 +156,20 @@ static int set_tcp_nodelay(socket_t socket)
sizeof(opt));
}
+static int set_tcp_ccalgo(socket_t socket, const char *ccalgo)
+{
+#ifdef HAVE_TCP_CONGESTION
+ return setsockopt(socket,
+ IPPROTO_TCP,
+ TCP_CONGESTION,
+ (void *)ccalgo,
+ strlen(ccalgo));
+#else
+ errno = ENOTSUP;
+ return -1;
+#endif
+}
+
/**
* @internal
*
@@ -256,6 +270,18 @@ socket_t ssh_connect_host_nonblocking(ssh_session session, const char *host,
}
}
+ if (session->opts.ccalgo) {
+ rc = set_tcp_ccalgo(s, session->opts.ccalgo);
+ if (rc < 0) {
+ ssh_set_error(session, SSH_FATAL,
+ "Failed to set TCP_CONGESTION on socket: %s",
+ ssh_strerror(errno, err_msg, SSH_ERRNO_MSG_MAX));
+ ssh_connect_socket_close(s);
+ s = -1;
+ continue;
+ }
+ }
+
errno = 0;
rc = connect(s, itr->ai_addr, itr->ai_addrlen);
if (rc == -1 && (errno != 0) && (errno != EINPROGRESS)) {
diff --git a/src/options.c b/src/options.c
index 38511455..a183605d 100644
--- a/src/options.c
+++ b/src/options.c
@@ -217,6 +217,7 @@ int ssh_options_copy(ssh_session src, ssh_session *dest)
new->opts.gss_delegate_creds = src->opts.gss_delegate_creds;
new->opts.flags = src->opts.flags;
new->opts.nodelay = src->opts.nodelay;
+ new->opts.ccalgo = src->opts.ccalgo;
new->opts.config_processed = src->opts.config_processed;
new->common.log_verbosity = src->common.log_verbosity;
new->common.callbacks = src->common.callbacks;
@@ -458,6 +459,10 @@ int ssh_options_set_algo(ssh_session session,
* Set it to disable Nagle's Algorithm (TCP_NODELAY) on the
* session socket. (int, 0=false)
*
+ * - SSH_OPTIONS_CCALGO
+ * Set it to specify TCP congestion control algorithm on the
+ * session socket (Linux only). (int, 0=false)
+ *
* - SSH_OPTIONS_PROCESS_CONFIG
* Set it to false to disable automatic processing of per-user
* and system-wide OpenSSH configuration files. LibSSH
@@ -1023,6 +1028,20 @@ int ssh_options_set(ssh_session session, enum ssh_options_e type,
session->opts.nodelay = (*x & 0xff) > 0 ? 1 : 0;
}
break;
+ case SSH_OPTIONS_CCALGO:
+ v = value;
+ if (v == NULL || v[0] == '\0') {
+ ssh_set_error_invalid(session);
+ return -1;
+ } else {
+ SAFE_FREE(session->opts.ccalgo);
+ session->opts.ccalgo = strdup(v);
+ if (session->opts.ccalgo == NULL) {
+ ssh_set_error_oom(session);
+ return -1;
+ }
+ }
+ break;
case SSH_OPTIONS_PROCESS_CONFIG:
if (value == NULL) {
ssh_set_error_invalid(session);
diff --git a/src/session.c b/src/session.c
index 8c509699..88602b6a 100644
--- a/src/session.c
+++ b/src/session.c
@@ -108,6 +108,7 @@ ssh_session ssh_new(void)
session->opts.fd = -1;
session->opts.compressionlevel = 7;
session->opts.nodelay = 0;
+ session->opts.ccalgo = NULL;
session->opts.flags = SSH_OPT_FLAG_PASSWORD_AUTH |
SSH_OPT_FLAG_PUBKEY_AUTH |
diff --git a/src/sftp.c b/src/sftp.c
index e01012a8..702623a0 100644
--- a/src/sftp.c
+++ b/src/sftp.c
@@ -2228,6 +2228,132 @@ ssize_t sftp_write(sftp_file file, const void *buf, size_t count) {
return -1; /* not reached */
}
+/*
+ * sftp_async_write is based on and sftp_async_write_end is copied from
+ * https://github.com/limes-datentechnik-gmbh/libssh
+ *
+ * sftp_async_write has some optimizations:
+ * - use ssh_buffer_new_size() to reduce realoc_buffer.
+ * - use ssh_buffer_add_func() to avoid memcpy from read buffer to ssh buffer.
+ */
+ssize_t sftp_async_write(sftp_file file, ssh_add_func f, size_t count, void *userdata,
+ uint32_t* id) {
+ sftp_session sftp = file->sftp;
+ ssh_buffer buffer;
+ uint32_t buf_sz;
+ ssize_t actual;
+ int len;
+ int packetlen;
+ int rc;
+
+#define HEADROOM 16
+ /* sftp_packet_write() prepends a 5-bytes (uint32_t length and
+ * 1-byte type) header to the head of the payload by
+ * ssh_buffer_prepend_data(). Inserting headroom by
+ * ssh_buffer_new_size() eliminates memcpy for prepending the
+ * header.
+ */
+
+ buf_sz = (HEADROOM + /* for header */
+ sizeof(uint32_t) + /* id */
+ ssh_string_len(file->handle) + 4 + /* file->handle */
+ sizeof(uint64_t) + /* file->offset */
+ sizeof(uint32_t) + /* count */
+ count); /* datastring */
+
+ buffer = ssh_buffer_new_size(buf_sz, HEADROOM);
+ if (buffer == NULL) {
+ ssh_set_error_oom(sftp->session);
+ return -1;
+ }
+
+ *id = sftp_get_new_id(file->sftp);
+
+ rc = ssh_buffer_pack(buffer,
+ "dSqd",
+ *id,
+ file->handle,
+ file->offset,
+ count); /* len of datastring */
+
+ if (rc != SSH_OK){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+
+ actual = ssh_buffer_add_func(buffer, f, count, userdata);
+ if (actual < 0){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+
+ packetlen=ssh_buffer_get_len(buffer)+5;
+ len = sftp_packet_write(file->sftp, SSH_FXP_WRITE, buffer);
+ ssh_buffer_free(buffer);
+ if (len < 0) {
+ return SSH_ERROR;
+ } else if (len != packetlen) {
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Could only send %d of %d bytes to remote host!", len, packetlen);
+ SSH_LOG(SSH_LOG_PACKET,
+ "Could not write as much data as expected");
+ return SSH_ERROR;
+ }
+
+ file->offset += actual;
+
+ return actual;
+}
+
+int sftp_async_write_end(sftp_file file, uint32_t id, int blocking) {
+ sftp_session sftp = file->sftp;
+ sftp_message msg = NULL;
+ sftp_status_message status;
+
+ msg = sftp_dequeue(sftp, id);
+ while (msg == NULL) {
+ if (!blocking && ssh_channel_poll(sftp->channel, 0) == 0) {
+ /* we cannot block */
+ return SSH_AGAIN;
+ }
+ if (sftp_read_and_dispatch(sftp) < 0) {
+ /* something nasty has happened */
+ return SSH_ERROR;
+ }
+ msg = sftp_dequeue(sftp, id);
+ }
+
+ switch (msg->packet_type) {
+ case SSH_FXP_STATUS:
+ status = parse_status_msg(msg);
+ sftp_message_free(msg);
+ if (status == NULL) {
+ return SSH_ERROR;
+ }
+ sftp_set_error(sftp, status->status);
+ switch (status->status) {
+ case SSH_FX_OK:
+ status_msg_free(status);
+ return SSH_OK;
+ default:
+ break;
+ }
+ ssh_set_error(sftp->session, SSH_REQUEST_DENIED,
+ "SFTP server: %s", status->errormsg);
+ status_msg_free(status);
+ return SSH_ERROR;
+ default:
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Received message %d during write!", msg->packet_type);
+ sftp_message_free(msg);
+ return SSH_ERROR;
+ }
+
+ return SSH_ERROR; /* not reached */
+}
+
/* Seek to a specific location in a file. */
int sftp_seek(sftp_file file, uint32_t new_offset) {
if (file == NULL) {
================================================
FILE: patch/libssh-0.11.2.patch
================================================
diff --git a/ConfigureChecks.cmake b/ConfigureChecks.cmake
index 8765dc6e..766e7d16 100644
--- a/ConfigureChecks.cmake
+++ b/ConfigureChecks.cmake
@@ -209,6 +209,7 @@ if (UNIX)
check_library_exists(util forkpty "" HAVE_LIBUTIL)
check_function_exists(cfmakeraw HAVE_CFMAKERAW)
check_function_exists(__strtoull HAVE___STRTOULL)
+ check_symbol_exists(TCP_CONGESTION "netinet/tcp.h" HAVE_TCP_CONGESTION)
endif (UNIX)
set(LIBSSH_REQUIRED_LIBRARIES ${_REQUIRED_LIBRARIES} CACHE INTERNAL "libssh required system libraries")
diff --git a/config.h.cmake b/config.h.cmake
index 8dce5273..ef534762 100644
--- a/config.h.cmake
+++ b/config.h.cmake
@@ -219,6 +219,8 @@
#cmakedefine HAVE_GCC_BOUNDED_ATTRIBUTE 1
+#cmakedefine HAVE_TCP_CONGESTION 1
+
/* Define to 1 if you want to enable GSSAPI */
#cmakedefine WITH_GSSAPI 1
diff --git a/include/libssh/buffer.h b/include/libssh/buffer.h
index d22178e7..2d6aa0a7 100644
--- a/include/libssh/buffer.h
+++ b/include/libssh/buffer.h
@@ -37,6 +37,8 @@ int ssh_buffer_add_u8(ssh_buffer buffer, uint8_t data);
int ssh_buffer_add_u16(ssh_buffer buffer, uint16_t data);
int ssh_buffer_add_u32(ssh_buffer buffer, uint32_t data);
int ssh_buffer_add_u64(ssh_buffer buffer, uint64_t data);
+ssize_t ssh_buffer_add_func(ssh_buffer buffer, ssh_add_func f, size_t max_bytes,
+ void *userdata);
int ssh_buffer_validate_length(struct ssh_buffer_struct *buffer, size_t len);
diff --git a/include/libssh/libssh.h b/include/libssh/libssh.h
index 3bddb019..1d5d7761 100644
--- a/include/libssh/libssh.h
+++ b/include/libssh/libssh.h
@@ -373,6 +373,7 @@ enum ssh_options_e {
SSH_OPTIONS_HOST,
SSH_OPTIONS_PORT,
SSH_OPTIONS_PORT_STR,
+ SSH_OPTIONS_AI_FAMILY,
SSH_OPTIONS_FD,
SSH_OPTIONS_USER,
SSH_OPTIONS_SSH_DIR,
@@ -407,6 +408,7 @@ enum ssh_options_e {
SSH_OPTIONS_GSSAPI_AUTH,
SSH_OPTIONS_GLOBAL_KNOWNHOSTS,
SSH_OPTIONS_NODELAY,
+ SSH_OPTIONS_CCALGO,
SSH_OPTIONS_PUBLICKEY_ACCEPTED_TYPES,
SSH_OPTIONS_PROCESS_CONFIG,
SSH_OPTIONS_REKEY_DATA,
@@ -876,6 +878,7 @@ LIBSSH_API const char* ssh_get_hmac_in(ssh_session session);
LIBSSH_API const char* ssh_get_hmac_out(ssh_session session);
LIBSSH_API ssh_buffer ssh_buffer_new(void);
+LIBSSH_API ssh_buffer ssh_buffer_new_size(uint32_t size, uint32_t headroom);
LIBSSH_API void ssh_buffer_free(ssh_buffer buffer);
#define SSH_BUFFER_FREE(x) \
do { if ((x) != NULL) { ssh_buffer_free(x); x = NULL; } } while(0)
@@ -886,6 +889,12 @@ LIBSSH_API void *ssh_buffer_get(ssh_buffer buffer);
LIBSSH_API uint32_t ssh_buffer_get_len(ssh_buffer buffer);
LIBSSH_API int ssh_session_set_disconnect_message(ssh_session session, const char *message);
+typedef ssize_t (*ssh_add_func) (void *ptr, size_t max_bytes, void *userdata);
+
+LIBSSH_API const char **ssh_ciphers(void);
+LIBSSH_API const char **ssh_hmacs(void);
+LIBSSH_API void ssh_use_openssh_proxy_jumps(int);
+
#ifndef LIBSSH_LEGACY_0_4
#include "libssh/legacy.h"
#endif
diff --git a/include/libssh/session.h b/include/libssh/session.h
index aed94072..327cf4fe 100644
--- a/include/libssh/session.h
+++ b/include/libssh/session.h
@@ -255,6 +255,7 @@ struct ssh_session_struct {
unsigned long timeout; /* seconds */
unsigned long timeout_usec;
uint16_t port;
+ int ai_family;
socket_t fd;
int StrictHostKeyChecking;
char compressionlevel;
@@ -264,6 +265,7 @@ struct ssh_session_struct {
int flags;
int exp_flags;
int nodelay;
+ char *ccalgo;
bool config_processed;
uint8_t options_seen[SOC_MAX];
uint64_t rekey_data;
diff --git a/include/libssh/sftp.h b/include/libssh/sftp.h
index cf4458c3..1a864795 100644
--- a/include/libssh/sftp.h
+++ b/include/libssh/sftp.h
@@ -569,6 +569,10 @@ SSH_DEPRECATED LIBSSH_API int sftp_async_read(sftp_file file,
uint32_t len,
uint32_t id);
+LIBSSH_API ssize_t sftp_async_write(sftp_file file, ssh_add_func f, size_t count,
+ void *userdata, uint32_t* id);
+LIBSSH_API int sftp_async_write_end(sftp_file file, uint32_t id, int blocking);
+
/**
* @brief Write to a file using an opened sftp file handle.
*
diff --git a/src/buffer.c b/src/buffer.c
index 449fa941..f49e8af6 100644
--- a/src/buffer.c
+++ b/src/buffer.c
@@ -142,6 +142,40 @@ struct ssh_buffer_struct *ssh_buffer_new(void)
return buf;
}
+/**
+ * @brief Create a new SSH buffer with a specified size and headroom.
+ *
+ * @param[in] len length for newly initialized SSH buffer.
+ * @param[in] headroom length for headroom
+ * @return A newly initialized SSH buffer, NULL on error.
+ */
+struct ssh_buffer_struct *ssh_buffer_new_size(uint32_t len, uint32_t headroom)
+{
+ struct ssh_buffer_struct *buf = NULL;
+ int rc;
+
+ if (len < headroom)
+ return NULL;
+
+ buf = calloc(1, sizeof(struct ssh_buffer_struct));
+ if (buf == NULL) {
+ return NULL;
+ }
+
+ rc = ssh_buffer_allocate_size(buf, len);
+ if (rc != 0) {
+ SAFE_FREE(buf);
+ return NULL;
+ }
+
+ buf->pos += headroom;
+ buf->used += headroom;
+
+ buffer_verify(buf);
+
+ return buf;
+}
+
/**
* @brief Deallocate a SSH buffer.
*
@@ -329,6 +363,49 @@ int ssh_buffer_add_data(struct ssh_buffer_struct *buffer, const void *data, uint
return 0;
}
+/**
+ * @brief Add data at the tail of a buffer by an external function
+ *
+ * @param[in] buffer The buffer to add data.
+ *
+ * @param[in] f function that adds data to the buffer.
+ *
+ * @param[in] max_bytes The maximum length of the data to add.
+ *
+ * @return actual bytes added on success, < 0 on error.
+ */
+ssize_t ssh_buffer_add_func(struct ssh_buffer_struct *buffer, ssh_add_func f,
+ size_t max_bytes, void *userdata)
+{
+ ssize_t actual;
+
+ if (buffer == NULL) {
+ return -1;
+ }
+
+ buffer_verify(buffer);
+
+ if (buffer->used + max_bytes < max_bytes) {
+ return -1;
+ }
+
+ if (buffer->allocated < (buffer->used + max_bytes)) {
+ if (buffer->pos > 0) {
+ buffer_shift(buffer);
+ }
+ if (realloc_buffer(buffer, buffer->used + max_bytes) < 0) {
+ return -1;
+ }
+ }
+
+ if ((actual = f(buffer->data + buffer->used, max_bytes, userdata)) < 0)
+ return -1;
+
+ buffer->used += actual;
+ buffer_verify(buffer);
+ return actual;
+}
+
/**
* @brief Ensure the buffer has at least a certain preallocated size.
*
diff --git a/src/connect.c b/src/connect.c
index 2cb64037..51f4c87e 100644
--- a/src/connect.c
+++ b/src/connect.c
@@ -109,7 +109,7 @@ static int ssh_connect_socket_close(socket_t s)
#endif
}
-static int getai(const char *host, int port, struct addrinfo **ai)
+static int getai(const char *host, int port, int ai_family, struct addrinfo **ai)
{
const char *service = NULL;
struct addrinfo hints;
@@ -118,7 +118,7 @@ static int getai(const char *host, int port, struct addrinfo **ai)
ZERO_STRUCT(hints);
hints.ai_protocol = IPPROTO_TCP;
- hints.ai_family = PF_UNSPEC;
+ hints.ai_family = ai_family > 0 ? ai_family : PF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
if (port == 0) {
@@ -151,6 +151,20 @@ static int set_tcp_nodelay(socket_t socket)
sizeof(opt));
}
+static int set_tcp_ccalgo(socket_t socket, const char *ccalgo)
+{
+#ifdef HAVE_TCP_CONGESTION
+ return setsockopt(socket,
+ IPPROTO_TCP,
+ TCP_CONGESTION,
+ (void *)ccalgo,
+ strlen(ccalgo));
+#else
+ errno = ENOTSUP;
+ return -1;
+#endif
+}
+
/**
* @internal
*
@@ -168,7 +182,7 @@ socket_t ssh_connect_host_nonblocking(ssh_session session, const char *host,
struct addrinfo *ai = NULL;
struct addrinfo *itr = NULL;
- rc = getai(host, port, &ai);
+ rc = getai(host, port, session->opts.ai_family, &ai);
if (rc != 0) {
ssh_set_error(session, SSH_FATAL,
"Failed to resolve hostname %s (%s)",
@@ -194,7 +208,7 @@ socket_t ssh_connect_host_nonblocking(ssh_session session, const char *host,
SSH_LOG(SSH_LOG_PACKET, "Resolving %s", bind_addr);
- rc = getai(bind_addr, 0, &bind_ai);
+ rc = getai(bind_addr, 0, session->opts.ai_family, &bind_ai);
if (rc != 0) {
ssh_set_error(session, SSH_FATAL,
"Failed to resolve bind address %s (%s)",
@@ -251,6 +265,18 @@ socket_t ssh_connect_host_nonblocking(ssh_session session, const char *host,
}
}
+ if (session->opts.ccalgo) {
+ rc = set_tcp_ccalgo(s, session->opts.ccalgo);
+ if (rc < 0) {
+ ssh_set_error(session, SSH_FATAL,
+ "Failed to set TCP_CONGESTION on socket: %s",
+ ssh_strerror(errno, err_msg, SSH_ERRNO_MSG_MAX));
+ ssh_connect_socket_close(s);
+ s = -1;
+ continue;
+ }
+ }
+
errno = 0;
rc = connect(s, itr->ai_addr, itr->ai_addrlen);
if (rc == -1) {
diff --git a/src/misc.c b/src/misc.c
index 774211fb..74e57959 100644
--- a/src/misc.c
+++ b/src/misc.c
@@ -71,6 +71,8 @@
#include "libssh/priv.h"
#include "libssh/misc.h"
#include "libssh/session.h"
+#include "libssh/wrapper.h"
+#include "libssh/crypto.h"
#ifdef HAVE_LIBGCRYPT
#define GCRYPT_STRING "/gcrypt"
@@ -2054,6 +2056,42 @@ ssize_t ssh_readn(int fd, void *buf, size_t nbytes)
return total_bytes_read;
}
+/**
+ * @brief Return supported cipher names
+ * @return The list of cipher names.
+ */
+const char **ssh_ciphers(void)
+{
+ struct ssh_cipher_struct *tab=ssh_get_ciphertab();
+ static const char *ciphers[32];
+ int n;
+
+ memset(ciphers, 0, sizeof(*ciphers));
+
+ for (n = 0; tab[n].name != NULL; n++) {
+ ciphers[n] = tab[n].name;
+ }
+ return ciphers;
+}
+
+/**
+ * @brief Return supported hmac names
+ * @return The list of hmac names.
+ */
+const char **ssh_hmacs(void)
+{
+ struct ssh_hmac_struct *tab=ssh_get_hmactab();
+ static const char *hmacs[32];
+ int n;
+
+ memset(hmacs, 0, sizeof(*hmacs));
+
+ for (n = 0; tab[n].name != NULL; n++) {
+ hmacs[n] = tab[n].name;
+ }
+ return hmacs;
+}
+
/**
* @brief Write the requested number of bytes to a local file.
*
@@ -2227,6 +2265,17 @@ ssh_proxyjumps_free(struct ssh_list *proxy_jump_list)
}
}
+static bool force_openssh_proxy_jumps;
+
+/**
+ * @breif set use openssh proxy jumps without the OPENSSH_PROXYJUMP env var
+ */
+void
+ssh_use_openssh_proxy_jumps(int v)
+{
+ force_openssh_proxy_jumps = (v > 0);
+}
+
/**
* @brief Check if libssh proxy jumps is enabled
*
@@ -2241,7 +2290,12 @@ ssh_libssh_proxy_jumps(void)
{
const char *t = getenv("OPENSSH_PROXYJUMP");
+ if (force_openssh_proxy_jumps)
+ return false;
+
return !(t != NULL && t[0] == '1');
}
+
+
/** @} */
diff --git a/src/options.c b/src/options.c
index 785296dd..a82d4d81 100644
--- a/src/options.c
+++ b/src/options.c
@@ -251,6 +251,7 @@ int ssh_options_copy(ssh_session src, ssh_session *dest)
new->opts.gss_delegate_creds = src->opts.gss_delegate_creds;
new->opts.flags = src->opts.flags;
new->opts.nodelay = src->opts.nodelay;
+ new->opts.ccalgo = src->opts.ccalgo;
new->opts.config_processed = src->opts.config_processed;
new->opts.control_master = src->opts.control_master;
new->common.log_verbosity = src->common.log_verbosity;
@@ -326,6 +327,9 @@ int ssh_options_set_algo(ssh_session session,
* - SSH_OPTIONS_PORT_STR:
* The port to connect to (const char *).
*
+ * - SSH_OPTIONS_AI_FAMILY:
+ * The address family for connecting (int *).
+ *
* - SSH_OPTIONS_FD:
* The file descriptor to use (socket_t).\n
* \n
@@ -571,6 +575,10 @@ int ssh_options_set_algo(ssh_session session,
* Set it to disable Nagle's Algorithm (TCP_NODELAY) on the
* session socket. (int, 0=false)
*
+ * - SSH_OPTIONS_CCALGO
+ * Set it to specify TCP congestion control algorithm on the
+ * session socket (Linux only). (int, 0=false)
+ *
* - SSH_OPTIONS_PROCESS_CONFIG
* Set it to false to disable automatic processing of per-user
* and system-wide OpenSSH configuration files. LibSSH
@@ -727,6 +735,21 @@ int ssh_options_set(ssh_session session, enum ssh_options_e type,
session->opts.port = i & 0xffffU;
}
break;
+ case SSH_OPTIONS_AI_FAMILY:
+ if (value == NULL) {
+ session->opts.ai_family = 0;
+ ssh_set_error_invalid(session);
+ return -1;
+ } else {
+ int *x = (int *) value;
+ if (*x < 0) {
+ session->opts.ai_family = 0;
+ ssh_set_error_invalid(session);
+ return -1;
+ }
+ session->opts.ai_family = *x;
+ }
+ break;
case SSH_OPTIONS_FD:
if (value == NULL) {
session->opts.fd = SSH_INVALID_SOCKET;
@@ -1241,6 +1264,20 @@ int ssh_options_set(ssh_session session, enum ssh_options_e type,
session->opts.nodelay = (*x & 0xff) > 0 ? 1 : 0;
}
break;
+ case SSH_OPTIONS_CCALGO:
+ v = value;
+ if (v == NULL || v[0] == '\0') {
+ ssh_set_error_invalid(session);
+ return -1;
+ } else {
+ SAFE_FREE(session->opts.ccalgo);
+ session->opts.ccalgo = strdup(v);
+ if (session->opts.ccalgo == NULL) {
+ ssh_set_error_oom(session);
+ return -1;
+ }
+ }
+ break;
case SSH_OPTIONS_PROCESS_CONFIG:
if (value == NULL) {
ssh_set_error_invalid(session);
diff --git a/src/session.c b/src/session.c
index 9fd5d946..ed9f908e 100644
--- a/src/session.c
+++ b/src/session.c
@@ -107,9 +107,11 @@ ssh_session ssh_new(void)
/* OPTIONS */
session->opts.StrictHostKeyChecking = 1;
session->opts.port = 22;
+ session->opts.ai_family = 0;
session->opts.fd = -1;
session->opts.compressionlevel = 7;
session->opts.nodelay = 0;
+ session->opts.ccalgo = NULL;
session->opts.identities_only = false;
session->opts.control_master = SSH_CONTROL_MASTER_NO;
diff --git a/src/sftp.c b/src/sftp.c
index 37b4133b..12b6d296 100644
--- a/src/sftp.c
+++ b/src/sftp.c
@@ -1488,6 +1488,132 @@ ssize_t sftp_write(sftp_file file, const void *buf, size_t count) {
return -1; /* not reached */
}
+/*
+ * sftp_async_write is based on and sftp_async_write_end is copied from
+ * https://github.com/limes-datentechnik-gmbh/libssh
+ *
+ * sftp_async_write has some optimizations:
+ * - use ssh_buffer_new_size() to reduce realoc_buffer.
+ * - use ssh_buffer_add_func() to avoid memcpy from read buffer to ssh buffer.
+ */
+ssize_t sftp_async_write(sftp_file file, ssh_add_func f, size_t count, void *userdata,
+ uint32_t* id) {
+ sftp_session sftp = file->sftp;
+ ssh_buffer buffer;
+ uint32_t buf_sz;
+ ssize_t actual;
+ int len;
+ int packetlen;
+ int rc;
+
+#define HEADROOM 16
+ /* sftp_packet_write() prepends a 5-bytes (uint32_t length and
+ * 1-byte type) header to the head of the payload by
+ * ssh_buffer_prepend_data(). Inserting headroom by
+ * ssh_buffer_new_size() eliminates memcpy for prepending the
+ * header.
+ */
+
+ buf_sz = (HEADROOM + /* for header */
+ sizeof(uint32_t) + /* id */
+ ssh_string_len(file->handle) + 4 + /* file->handle */
+ sizeof(uint64_t) + /* file->offset */
+ sizeof(uint32_t) + /* count */
+ count); /* datastring */
+
+ buffer = ssh_buffer_new_size(buf_sz, HEADROOM);
+ if (buffer == NULL) {
+ ssh_set_error_oom(sftp->session);
+ return -1;
+ }
+
+ *id = sftp_get_new_id(file->sftp);
+
+ rc = ssh_buffer_pack(buffer,
+ "dSqd",
+ *id,
+ file->handle,
+ file->offset,
+ count); /* len of datastring */
+
+ if (rc != SSH_OK){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+
+ actual = ssh_buffer_add_func(buffer, f, count, userdata);
+ if (actual < 0){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+
+ packetlen=ssh_buffer_get_len(buffer)+5;
+ len = sftp_packet_write(file->sftp, SSH_FXP_WRITE, buffer);
+ ssh_buffer_free(buffer);
+ if (len < 0) {
+ return SSH_ERROR;
+ } else if (len != packetlen) {
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Could only send %d of %d bytes to remote host!", len, packetlen);
+ SSH_LOG(SSH_LOG_PACKET,
+ "Could not write as much data as expected");
+ return SSH_ERROR;
+ }
+
+ file->offset += actual;
+
+ return actual;
+}
+
+int sftp_async_write_end(sftp_file file, uint32_t id, int blocking) {
+ sftp_session sftp = file->sftp;
+ sftp_message msg = NULL;
+ sftp_status_message status;
+
+ msg = sftp_dequeue(sftp, id);
+ while (msg == NULL) {
+ if (!blocking && ssh_channel_poll(sftp->channel, 0) == 0) {
+ /* we cannot block */
+ return SSH_AGAIN;
+ }
+ if (sftp_read_and_dispatch(sftp) < 0) {
+ /* something nasty has happened */
+ return SSH_ERROR;
+ }
+ msg = sftp_dequeue(sftp, id);
+ }
+
+ switch (msg->packet_type) {
+ case SSH_FXP_STATUS:
+ status = parse_status_msg(msg);
+ sftp_message_free(msg);
+ if (status == NULL) {
+ return SSH_ERROR;
+ }
+ sftp_set_error(sftp, status->status);
+ switch (status->status) {
+ case SSH_FX_OK:
+ status_msg_free(status);
+ return SSH_OK;
+ default:
+ break;
+ }
+ ssh_set_error(sftp->session, SSH_REQUEST_DENIED,
+ "SFTP server: %s", status->errormsg);
+ status_msg_free(status);
+ return SSH_ERROR;
+ default:
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Received message %d during write!", msg->packet_type);
+ sftp_message_free(msg);
+ return SSH_ERROR;
+ }
+
+ return SSH_ERROR; /* not reached */
+}
+
/* Seek to a specific location in a file. */
int sftp_seek(sftp_file file, uint32_t new_offset) {
if (file == NULL) {
================================================
FILE: patch/libssh-0.9.6.patch
================================================
diff --git a/DefineOptions.cmake b/DefineOptions.cmake
index b82a5018..f1f2ab9d 100644
--- a/DefineOptions.cmake
+++ b/DefineOptions.cmake
@@ -15,13 +15,14 @@ option(UNIT_TESTING "Build with unit tests" OFF)
option(CLIENT_TESTING "Build with client tests; requires openssh" OFF)
option(SERVER_TESTING "Build with server tests; requires openssh and dropbear" OFF)
option(WITH_BENCHMARKS "Build benchmarks tools" OFF)
-option(WITH_EXAMPLES "Build examples" ON)
+option(WITH_EXAMPLES "Build examples" OFF)
option(WITH_NACL "Build with libnacl (curve25519)" ON)
option(WITH_SYMBOL_VERSIONING "Build with symbol versioning" ON)
option(WITH_ABI_BREAK "Allow ABI break" OFF)
option(WITH_GEX "Enable DH Group exchange mechanisms" ON)
option(FUZZ_TESTING "Build with fuzzer for the server" OFF)
option(PICKY_DEVELOPER "Build with picky developer flags" OFF)
+option(WITH_STATIC_LIB "Build static library" ON)
if (WITH_ZLIB)
set(WITH_LIBZ ON)
@@ -53,3 +54,7 @@ endif (NOT GLOBAL_BIND_CONFIG)
if (NOT GLOBAL_CLIENT_CONFIG)
set(GLOBAL_CLIENT_CONFIG "/etc/ssh/ssh_config")
endif (NOT GLOBAL_CLIENT_CONFIG)
+
+if (WITH_STATIC_LIB)
+ set(BUILD_STATIC_LIB ON)
+endif()
diff --git a/include/libssh/sftp.h b/include/libssh/sftp.h
index 8c14b21d..95ac1d6b 100644
--- a/include/libssh/sftp.h
+++ b/include/libssh/sftp.h
@@ -565,6 +565,9 @@ LIBSSH_API int sftp_async_read(sftp_file file, void *data, uint32_t len, uint32_
*/
LIBSSH_API ssize_t sftp_write(sftp_file file, const void *buf, size_t count);
+LIBSSH_API int sftp_async_write(sftp_file file, const void *buf, size_t count, uint32_t* id);
+LIBSSH_API int sftp_async_write_end(sftp_file file, uint32_t id, int blocking);
+
/**
* @brief Seek to a specific location in a file.
*
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index a576cf71..303a1c7f 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -412,6 +412,10 @@ if (BUILD_STATIC_LIB)
if (WIN32)
target_compile_definitions(ssh-static PUBLIC "LIBSSH_STATIC")
endif (WIN32)
+ install(TARGETS ssh-static
+ EXPORT libssh-config
+ LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
+ COMPONENT libraries)
endif (BUILD_STATIC_LIB)
message(STATUS "Threads_FOUND=${Threads_FOUND}")
diff --git a/src/sftp.c b/src/sftp.c
index a8346040..a4261ec9 100644
--- a/src/sftp.c
+++ b/src/sftp.c
@@ -2234,6 +2234,102 @@ ssize_t sftp_write(sftp_file file, const void *buf, size_t count) {
return -1; /* not reached */
}
+/*
+ * sftp_async_write and sftp_async_write_end are copied from
+ * https://github.com/limes-datentechnik-gmbh/libssh
+ */
+int sftp_async_write(sftp_file file, const void *buf, size_t count, uint32_t* id) {
+ sftp_session sftp = file->sftp;
+ ssh_buffer buffer;
+ int len;
+ int packetlen;
+ int rc;
+
+ buffer = ssh_buffer_new();
+ if (buffer == NULL) {
+ ssh_set_error_oom(sftp->session);
+ return -1;
+ }
+
+ *id = sftp_get_new_id(file->sftp);
+
+ rc = ssh_buffer_pack(buffer,
+ "dSqdP",
+ *id,
+ file->handle,
+ file->offset,
+ count, /* len of datastring */
+ (size_t)count, buf);
+ if (rc != SSH_OK){
+ ssh_set_error_oom(sftp->session);
+ ssh_buffer_free(buffer);
+ return SSH_ERROR;
+ }
+ packetlen=ssh_buffer_get_len(buffer)+5;
+ len = sftp_packet_write(file->sftp, SSH_FXP_WRITE, buffer);
+ ssh_buffer_free(buffer);
+ if (len < 0) {
+ return SSH_ERROR;
+ } else if (len != packetlen) {
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Could only send %d of %d bytes to remote host!", len, packetlen);
+ SSH_LOG(SSH_LOG_PACKET,
+ "Could not write as much data as expected");
+ return SSH_ERROR;
+ }
+
+ file->offset += count;
+
+ return SSH_OK;
+}
+
+int sftp_async_write_end(sftp_file file, uint32_t id, int blocking) {
+ sftp_session sftp = file->sftp;
+ sftp_message msg = NULL;
+ sftp_status_message status;
+
+ msg = sftp_dequeue(sftp, id);
+ while (msg == NULL) {
+ if (!blocking && ssh_channel_poll(sftp->channel, 0) == 0) {
+ /* we cannot block */
+ return SSH_AGAIN;
+ }
+ if (sftp_read_and_dispatch(sftp) < 0) {
+ /* something nasty has happened */
+ return SSH_ERROR;
+ }
+ msg = sftp_dequeue(sftp, id);
+ }
+
+ switch (msg->packet_type) {
+ case SSH_FXP_STATUS:
+ status = parse_status_msg(msg);
+ sftp_message_free(msg);
+ if (status == NULL) {
+ return SSH_ERROR;
+ }
+ sftp_set_error(sftp, status->status);
+ switch (status->status) {
+ case SSH_FX_OK:
+ status_msg_free(status);
+ return SSH_OK;
+ default:
+ break;
+ }
+ ssh_set_error(sftp->session, SSH_REQUEST_DENIED,
+ "SFTP server: %s", status->errormsg);
+ status_msg_free(status);
+ return SSH_ERROR;
+ default:
+ ssh_set_error(sftp->session, SSH_FATAL,
+ "Received message %d during write!", msg->packet_type);
+ sftp_message_free(msg);
+ return SSH_ERROR;
+ }
+
+ return SSH_ERROR; /* not reached */
+}
+
/* Seek to a specific location in a file. */
int sftp_seek(sftp_file file, uint32_t new_offset) {
if (file == NULL) {
================================================
FILE: rpm/.gitignore
================================================
# generated by cmake
mscp.spec
================================================
FILE: rpm/mscp.spec.in
================================================
Name: mscp
Version: @MSCP_VERSION@
Release: 1%{?dist}
Summary: mscp, fast file transfer over multiple SSH connections
Group: Applications/Internet
License: GPLv3
URL: https://github.com/upa/mscp
Source0: %{name}-%{version}.tar.gz
BuildRequires: gcc make cmake zlib-devel openssl-devel
Requires: glibc crypto-policies krb5-libs openssl-libs libcom_err
%description
mscp transfers files over multiple SSH connections. Multiple threads
and connections in mscp transfer (1) multiple files simultaneously
and (2) a large file in parallel. It would shorten the waiting time
for transferring a lot of/large files over networks.
%global debug_package %{nil}
%prep
%setup -q
%build
cmake -S . -B build -DINSTALL_EXECUTABLE_ONLY=ON
make -C build %{?_smp_mflags}
%install
make -C build install DESTDIR=%{buildroot}
%files
/usr/local/bin/mscp
/usr/local/share/man/man1/mscp.1
%changelog
* Sat Nov 08 2025 Ryo Nakamura <upa@haeena.net> - 0.2.4-1
- RPM release for v0.2.4
* Tue Aug 12 2025 Ryo Nakamura <upa@haeena.net> - 0.2.3-1
- RPM release for v0.2.3
* Wed Apr 16 2025 Ryo Nakamura <upa@haeena.net> - 0.2.2-1
- RPM release for v0.2.2
* Sat May 11 2024 Ryo Nakamura <upa@haeena.net> - 0.2.1-1
- RPM release for v0.2.1
* Mon Apr 15 2024 Ryo Nakamura <upa@haeena.net> - 0.2.0-1
- RPM release for v0.2.0
* Thu Mar 14 2024 Ryo Nakamura <upa@haeena.net> - 0.1.5-0
- RPM release for v0.1.5
* Wed Feb 07 2024 Ryo Nakamura <upa@haeena.net> - 0.1.4-0
- RPM release for v0.1.4
* Sat Feb 03 2024 Ryo Nakamura <upa@haeena.net> - 0.1.3-0
- Initial release for rpm packaging
================================================
FILE: scripts/install-build-deps.sh
================================================
#!/usr/bin/env bash
#
# Install build dpenedencies.
set -e
#set -u
function print_help() {
echo "$0 [options]"
echo " --dont-install Print required packages."
echo " --platform [PLATFORM] PLATFORM is Kernel-ID, e.g., Linux-ubuntu."
echo " Automatically detected if not specified."
}
platform=$(uname -s)
doinstall=1
if [ -e /etc/os-release ]; then
source /etc/os-release
platform=${platform}-${ID}
fi
while getopts h-: opt; do
optarg="${!OPTIND}"
[[ "$opt" = - ]] && opt="-$OPTARG"
case "-${opt}" in
--dont-install)
doinstall=0
;;
--platform)
platform=$optarg
shift
;;
-h)
print_help
exit 0
;;
*)
print_help
exit 1
;;
esac
done
case $platform in
Darwin)
cmd="brew install"
pkgs="openssl@3"
;;
Linux-ubuntu* | Linux-debian* | Linux-devuan*)
cmd="apt-get install --no-install-recommends -y"
pkgs="gcc make cmake zlib1g-dev libssl-dev libkrb5-dev"
;;
Linux-centos* | Linux-rhel* | Linux-rocky* | Linux-almalinux)
cmd="yum install -y"
pkgs="gcc make cmake zlib-devel openssl-devel rpm-build"
;;
Linux-arch*)
cmd="pacman --no-confirm -S"
pkgs="gcc make cmake"
;;
FreeBSD-freebsd)
cmd="pkg install"
pkgs="cmake"
;;
*)
echo "unsupported platform: $platform"
exit 1
;;
esac
if [ $doinstall -gt 0 ]; then
echo do "$cmd $pkgs"
$cmd $pkgs
else
echo $pkgs
fi
================================================
FILE: scripts/test-in-container.sh
================================================
#!/bin/bash -e
#
# Run this script in mscp docker containers.
# This script runs end-to-end test with installed mscp.
script_dir=$(cd $(dirname ${0}) && pwd)
cd $script_dir
set -x
# sshd Linsten on 22 and 8022
echo "Port 22" >> /etc/ssh/sshd_config
echo "Port 8022" >> /etc/ssh/sshd_config
## Alpine default sshd disables TcpForwarding, which is required for proxyjump test
sed -i -e 's/AllowTcpForwarding no/AllowTcpForwarding yes/' /etc/ssh/sshd_config
# Run sshd
if [ ! -e /var/run/sshd.pid ]; then
/usr/sbin/sshd -E /tmp/sshd.log
sleep 1
fi
for port in 22 8022; do
ssh-keyscan -p $port localhost >> ${HO
gitextract_bck5e2hc/
├── .clang-format
├── .dockerignore
├── .github/
│ └── workflows/
│ ├── build-freebsd.yml
│ ├── build-macos.yml
│ ├── build-ubuntu.yml
│ ├── codeql.yml
│ ├── release.yml
│ └── test.yml
├── .gitignore
├── .gitmodules
├── CMakeLists.txt
├── Dockerfile/
│ ├── README.md
│ ├── almalinux-9.3.Dockerfile
│ ├── alpine-3.22.Dockerfile
│ ├── arch-base.Dockerfile
│ ├── build-deb.Dockerfile
│ ├── build-srpm.Dockerfile
│ ├── rocky-8.9.Dockerfile
│ ├── rocky-9.3.Dockerfile
│ ├── ubuntu-20.04.Dockerfile
│ ├── ubuntu-22.04.Dockerfile
│ └── ubuntu-24.04.Dockerfile
├── Doxyfile
├── LICENSE
├── README.md
├── VERSION
├── conanfile.txt
├── doc/
│ ├── RELEASE.md
│ ├── mscp.1.in
│ └── mscp.rst
├── include/
│ ├── config.h.in
│ └── mscp.h
├── patch/
│ ├── README.md
│ ├── libssh-0.10.4.patch
│ ├── libssh-0.10.6-2-g6f1b1e76.patch
│ ├── libssh-0.10.6.patch
│ ├── libssh-0.11.2.patch
│ └── libssh-0.9.6.patch
├── rpm/
│ ├── .gitignore
│ └── mscp.spec.in
├── scripts/
│ ├── install-build-deps.sh
│ └── test-in-container.sh
├── src/
│ ├── atomic.h
│ ├── bwlimit.c
│ ├── bwlimit.h
│ ├── checkpoint.c
│ ├── checkpoint.h
│ ├── fileops.c
│ ├── fileops.h
│ ├── main.c
│ ├── minmax.h
│ ├── mscp.c
│ ├── openbsd-compat/
│ │ ├── openbsd-compat.h
│ │ └── strlcat.c
│ ├── path.c
│ ├── path.h
│ ├── platform.c
│ ├── platform.h
│ ├── pool.c
│ ├── pool.h
│ ├── print.c
│ ├── print.h
│ ├── ssh.c
│ ├── ssh.h
│ ├── strerrno.c
│ └── strerrno.h
└── test/
├── .gitignore
├── README.md
├── conftest.py
├── test_e2e.py
└── util.py
SYMBOL INDEX (280 symbols across 23 files)
FILE: include/mscp.h
type mscp_opts (line 40) | struct mscp_opts {
type mscp_ssh_opts (line 59) | struct mscp_ssh_opts {
type mscp_stats (line 95) | struct mscp_stats {
type mscp (line 102) | struct mscp
type mscp (line 112) | struct mscp
type mscp_opts (line 112) | struct mscp_opts
type mscp_ssh_opts (line 112) | struct mscp_ssh_opts
type mscp (line 122) | struct mscp
type mscp (line 134) | struct mscp
type mscp (line 147) | struct mscp
type mscp (line 158) | struct mscp
type mscp (line 172) | struct mscp
type mscp (line 182) | struct mscp
type mscp (line 207) | struct mscp
type mscp (line 217) | struct mscp
type mscp (line 231) | struct mscp
type mscp (line 239) | struct mscp
type mscp (line 250) | struct mscp
type mscp (line 258) | struct mscp
type mscp_stats (line 258) | struct mscp_stats
type mscp (line 270) | struct mscp
type mscp (line 279) | struct mscp
FILE: src/atomic.h
type refcnt (line 9) | typedef int refcnt;
function refcnt_inc (line 11) | static inline void refcnt_inc(refcnt *cnt)
function refcnt (line 16) | static inline refcnt refcnt_dec(refcnt *cnt)
type pthread_mutex_t (line 23) | typedef pthread_mutex_t lock;
function lock_init (line 25) | static inline void lock_init(lock *l)
function lock_acquire (line 30) | static inline void lock_acquire(lock *l)
function lock_release (line 36) | static inline void lock_release(lock *l)
function lock_release_via_cleanup (line 42) | static inline void lock_release_via_cleanup(void *l)
type pthread_rwlock_t (line 54) | typedef pthread_rwlock_t rwlock;
function rwlock_init (line 56) | static inline void rwlock_init(rwlock *rw)
function rwlock_read_acquire (line 61) | static inline void rwlock_read_acquire(rwlock *rw)
function rwlock_write_acquire (line 67) | static inline void rwlock_write_acquire(rwlock *rw)
function rwlock_release (line 73) | static inline void rwlock_release(rwlock *rw)
function rwlock_release_via_cleanup (line 79) | static inline void rwlock_release_via_cleanup(void *rw)
FILE: src/bwlimit.c
function bwlimit_init (line 13) | int bwlimit_init(struct bwlimit *bw, uint64_t bps, uint64_t win)
function bwlimit_wait (line 53) | int bwlimit_wait(struct bwlimit *bw, size_t nr_bytes)
FILE: src/bwlimit.h
type bwlimit (line 11) | struct bwlimit {
type bwlimit (line 21) | struct bwlimit
type bwlimit (line 25) | struct bwlimit
FILE: src/checkpoint.c
type checkpoint_file_hdr (line 121) | struct checkpoint_file_hdr {
type checkpoint_obj_hdr (line 126) | struct checkpoint_obj_hdr {
type checkpoint_obj_meta (line 132) | struct checkpoint_obj_meta {
type checkpoint_obj_path (line 139) | struct checkpoint_obj_path {
type checkpoint_obj_chunk (line 161) | struct checkpoint_obj_chunk {
function checkpoint_write_path (line 171) | static int checkpoint_write_path(int fd, struct path *p, unsigned int idx)
function checkpoint_write_chunk (line 205) | static int checkpoint_write_chunk(int fd, struct chunk *c)
function checkpoint_save (line 224) | int checkpoint_save(const char *pathname, int dir, const char *user, con...
function checkpoint_load_meta (line 299) | static int checkpoint_load_meta(struct checkpoint_obj_hdr *hdr, char *re...
function checkpoint_load_path (line 319) | static int checkpoint_load_path(struct checkpoint_obj_hdr *hdr, pool *pa...
function checkpoint_load_chunk (line 358) | static int checkpoint_load_chunk(struct checkpoint_obj_hdr *hdr, pool *p...
function checkpoint_read_obj (line 385) | static int checkpoint_read_obj(int fd, void *buf, size_t count)
function checkpoint_read_file_hdr (line 419) | static int checkpoint_read_file_hdr(int fd)
function checkpoint_load (line 447) | static int checkpoint_load(const char *pathname, char *remote, size_t le...
function checkpoint_load_remote (line 497) | int checkpoint_load_remote(const char *pathname, char *remote, size_t le...
function checkpoint_load_paths (line 502) | int checkpoint_load_paths(const char *pathname, pool *path_pool, pool *c...
FILE: src/fileops.c
function set_tls_sftp_session (line 18) | void set_tls_sftp_session(sftp_session sftp)
function sftp_err_to_errno (line 23) | static void sftp_err_to_errno(sftp_session sftp)
function MDIR (line 69) | MDIR *mscp_opendir(const char *path, sftp_session sftp)
function MDIR (line 97) | MDIR *mscp_opendir_wrapped(const char *path)
function mscp_closedir (line 102) | void mscp_closedir(MDIR *md)
type dirent (line 112) | struct dirent
type dirent (line 117) | struct dirent
type dirent (line 120) | struct dirent
function mscp_mkdir (line 143) | int mscp_mkdir(const char *path, mode_t mode, sftp_session sftp)
function sftp_attr_to_stat (line 160) | static void sftp_attr_to_stat(sftp_attributes attr, struct stat *st)
function mscp_stat (line 201) | int mscp_stat(const char *path, struct stat *st, sftp_session sftp)
function mscp_stat_wrapped (line 223) | int mscp_stat_wrapped(const char *path, struct stat *st)
function mscp_lstat (line 228) | int mscp_lstat(const char *path, struct stat *st, sftp_session sftp)
function mscp_lstat_wrapped (line 248) | int mscp_lstat_wrapped(const char *path, struct stat *st)
function mf (line 253) | mf *mscp_open(const char *path, int flags, mode_t mode, sftp_session sftp)
function mscp_close (line 281) | void mscp_close(mf *f)
function off_t (line 290) | off_t mscp_lseek(mf *f, off_t off)
function mscp_setstat (line 303) | int mscp_setstat(const char *path, struct stat *st, bool preserve_ts, sf...
function mscp_glob (line 338) | int mscp_glob(const char *pattern, int flags, glob_t *pglob, sftp_sessio...
function mscp_globfree (line 378) | void mscp_globfree(glob_t *pglob)
FILE: src/fileops.h
type mdir_struct (line 16) | struct mdir_struct {
type MDIR (line 20) | typedef struct mdir_struct MDIR;
type dirent (line 25) | struct dirent
type stat (line 30) | struct stat
type stat (line 31) | struct stat
type stat (line 33) | struct stat
type stat (line 34) | struct stat
type mf_struct (line 38) | struct mf_struct {
type mf (line 42) | typedef struct mf_struct mf;
type stat (line 51) | struct stat
FILE: src/main.c
function usage (line 24) | void usage(bool print_help)
type target (line 153) | struct target {
function compare_remote (line 160) | int compare_remote(struct target *a, struct target *b)
type target (line 192) | struct target
type target (line 205) | struct target
type target (line 208) | struct target
type mscp (line 282) | struct mscp
function sigint_handler (line 286) | void sigint_handler(int sig)
function print_cli (line 294) | void print_cli(const char *fmt, ...)
function atol_with_unit (line 305) | long atol_with_unit(char *value, bool i)
function to_dev_null (line 336) | int to_dev_null(int fd)
function main (line 354) | int main(int argc, char **argv)
function calculate_timedelta (line 624) | double calculate_timedelta(struct timeval *b, struct timeval *a)
function calculate_bps (line 640) | double calculate_bps(size_t diff, struct timeval *b, struct timeval *a)
type timeval (line 645) | struct timeval
type timeval (line 645) | struct timeval
function print_progress_bar (line 684) | void print_progress_bar(double percent, char *suffix)
function print_progress (line 727) | void print_progress(struct timeval *b, struct timeval *a, size_t total, ...
type xfer_stat (line 769) | struct xfer_stat {
type xfer_stat (line 775) | struct xfer_stat
function print_stat (line 777) | void print_stat(bool final)
FILE: src/mscp.c
type mscp_thread (line 24) | struct mscp_thread {
type mscp (line 38) | struct mscp {
function expand_coremask (line 79) | static int expand_coremask(const char *coremask, int **cores, int *nr_co...
function default_nr_threads (line 140) | static int default_nr_threads()
function validate_and_set_defaut_params (line 145) | static int validate_and_set_defaut_params(struct mscp_opts *o)
function mscp_set_remote (line 193) | int mscp_set_remote(struct mscp *m, const char *remote_host, int direction)
type mscp (line 214) | struct mscp
type mscp_opts (line 214) | struct mscp_opts
type mscp_ssh_opts (line 214) | struct mscp_ssh_opts
type mscp (line 216) | struct mscp
function mscp_connect (line 301) | int mscp_connect(struct mscp *m)
function mscp_add_src_path (line 310) | int mscp_add_src_path(struct mscp *m, const char *src_path)
function mscp_set_dst_path (line 324) | int mscp_set_dst_path(struct mscp *m, const char *dst_path)
function get_page_mask (line 341) | static size_t get_page_mask(void)
function mscp_stop_copy_thread (line 347) | static void mscp_stop_copy_thread(struct mscp *m)
function mscp_stop_scan_thread (line 359) | static void mscp_stop_scan_thread(struct mscp *m)
function mscp_stop (line 365) | void mscp_stop(struct mscp *m)
type mscp_thread (line 373) | struct mscp_thread
type mscp (line 374) | struct mscp
type path_resolve_args (line 376) | struct path_resolve_args
type path (line 377) | struct path
type stat (line 378) | struct stat
function mscp_scan (line 460) | int mscp_scan(struct mscp *m)
function mscp_scan_join (line 485) | int mscp_scan_join(struct mscp *m)
function mscp_checkpoint_get_remote (line 496) | int mscp_checkpoint_get_remote(const char *pathname, char *remote, size_...
function mscp_checkpoint_load (line 501) | int mscp_checkpoint_load(struct mscp *m, const char *pathname)
function mscp_checkpoint_save (line 520) | int mscp_checkpoint_save(struct mscp *m, const char *pathname)
type mscp_thread (line 528) | struct mscp_thread
type mscp (line 528) | struct mscp
type mscp_thread (line 530) | struct mscp_thread
function mscp_start (line 555) | int mscp_start(struct mscp *m)
function mscp_join (line 581) | int mscp_join(struct mscp *m)
function wait_for_interval (line 628) | static void wait_for_interval(int interval)
type mscp_thread (line 646) | struct mscp_thread
type mscp (line 647) | struct mscp
type chunk (line 648) | struct chunk
function mscp_cleanup (line 738) | void mscp_cleanup(struct mscp *m)
function mscp_free (line 751) | void mscp_free(struct mscp *m)
function mscp_get_stats (line 765) | void mscp_get_stats(struct mscp *m, struct mscp_stats *s)
FILE: src/openbsd-compat/strlcat.c
function strlcat (line 34) | size_t
FILE: src/path.c
type path_resolve_args (line 18) | struct path_resolve_args
type chunk (line 81) | struct chunk
type path (line 81) | struct path
type chunk (line 83) | struct chunk
function resolve_chunk (line 99) | static int resolve_chunk(struct path *p, size_t size, struct path_resolv...
function free_path (line 135) | void free_path(struct path *p)
type path (line 144) | struct path
type path (line 146) | struct path
function append_path (line 163) | static int append_path(sftp_session sftp, const char *path, struct stat st,
function check_path_should_skip (line 200) | static bool check_path_should_skip(const char *path)
function walk_path_recursive (line 210) | static int walk_path_recursive(sftp_session sftp, const char *path,
function walk_src_path (line 262) | int walk_src_path(sftp_session src_sftp, const char *src_path,
function touch_dst_path (line 270) | static int touch_dst_path(struct path *p, sftp_session sftp)
function prepare_dst_path (line 320) | static int prepare_dst_path(struct path *p, sftp_session dst_sftp)
function read_to_buf (line 341) | static ssize_t read_to_buf(void *ptr, size_t len, void *userdata)
function copy_chunk_l2r (line 347) | static int copy_chunk_l2r(struct chunk *c, int fd, sftp_file sf, int nr_...
function copy_chunk_r2l (line 413) | static int copy_chunk_r2l(struct chunk *c, sftp_file sf, int fd,
function _copy_chunk (line 479) | static int _copy_chunk(struct chunk *c, mf *s, mf *d, int nr_ahead, int ...
function copy_chunk (line 493) | int copy_chunk(struct chunk *c, sftp_session src_sftp, sftp_session dst_...
FILE: src/path.h
type path (line 14) | struct path {
type path (line 28) | struct path
type chunk (line 30) | struct chunk {
type chunk (line 40) | struct chunk
type path (line 40) | struct path
type path_resolve_args (line 42) | struct path_resolve_args {
type path_resolve_args (line 63) | struct path_resolve_args
type path (line 66) | struct path
type chunk (line 69) | struct chunk
type bwlimit (line 70) | struct bwlimit
FILE: src/platform.c
function nr_cpus (line 31) | int nr_cpus()
function set_thread_affinity (line 44) | int set_thread_affinity(pthread_t tid, int core)
function setutimes (line 50) | int setutimes(const char *path, struct timespec atime, struct timespec m...
function random_string (line 65) | static void random_string(char *buf, size_t size)
function sem_t (line 77) | sem_t *sem_create(int value)
function sem_release (line 91) | int sem_release(sem_t *sem)
function nr_cpus (line 99) | int nr_cpus()
function nr_cpus (line 109) | int nr_cpus()
function set_thread_affinity (line 118) | int set_thread_affinity(pthread_t tid, int core)
function setutimes (line 132) | int setutimes(const char *path, struct timespec atime, struct timespec m...
function sem_t (line 145) | sem_t *sem_create(int value)
function sem_release (line 160) | int sem_release(sem_t *sem)
FILE: src/platform.h
type timespec (line 13) | struct timespec
type timespec (line 13) | struct timespec
FILE: src/pool.c
function pool (line 8) | pool *pool_new(void)
function pool_free (line 28) | void pool_free(pool *p)
function pool_zeroize (line 37) | void pool_zeroize(pool *p, pool_map_f f)
function pool_destroy (line 46) | void pool_destroy(pool *p, pool_map_f f)
function pool_push (line 52) | int pool_push(pool *p, void *v)
function pool_push_lock (line 69) | int pool_push_lock(pool *p, void *v)
function pool_iter_has_next_lock (line 116) | bool pool_iter_has_next_lock(pool *p)
FILE: src/pool.h
type pool_struct (line 14) | struct pool_struct {
type pool (line 22) | typedef struct pool_struct pool;
FILE: src/print.c
function set_print_severity (line 8) | void set_print_severity(int serverity)
function get_print_severity (line 16) | int get_print_severity()
FILE: src/ssh.c
function ssh_set_opts (line 17) | static int ssh_set_opts(ssh_session ssh, struct mscp_ssh_opts *opts)
function ssh_authenticate (line 115) | static int ssh_authenticate(ssh_session ssh, struct mscp_ssh_opts *opts)
function ssh_cache_passphrase (line 162) | static int ssh_cache_passphrase(const char *prompt, char *buf, size_t le...
type ssh_callbacks_struct (line 191) | struct ssh_callbacks_struct
function ssh_session (line 196) | static ssh_session ssh_init_session(const char *sshdst, struct mscp_ssh_...
function sftp_session (line 236) | sftp_session ssh_init_sftp_session(const char *sshdst, struct mscp_ssh_o...
function ssh_verify_known_hosts (line 264) | static int ssh_verify_known_hosts(ssh_session session)
function ssh_authenticate_kbdint (line 350) | static int ssh_authenticate_kbdint(ssh_session ssh)
function ssh_sftp_close (line 398) | void ssh_sftp_close(sftp_session sftp)
FILE: src/ssh.h
type mscp_ssh_opts (line 14) | struct mscp_ssh_opts
FILE: src/strerrno.c
function priv_set_err (line 23) | void priv_set_err(const char *fmt, ...)
FILE: test/conftest.py
function pytest_addoption (line 3) | def pytest_addoption(parser):
function mscp (line 8) | def mscp(request):
FILE: test/test_e2e.py
function run2ok (line 19) | def run2ok(args, env = None, quiet = False):
function run2ng (line 25) | def run2ng(args, env = None, timeout = None, quiet = False):
function cleanup_files (line 36) | def cleanup_files():
function test_usage (line 77) | def test_usage(mscp):
function test_invalid_chunk_size_config (line 81) | def test_invalid_chunk_size_config(mscp):
function test_nonidentical_hostnames (line 90) | def test_nonidentical_hostnames(mscp, args):
function test_single_copy (line 111) | def test_single_copy(mscp, src_prefix, dst_prefix, src, dst):
function test_failed_to_copy_nonexistent_file (line 118) | def test_failed_to_copy_nonexistent_file(mscp, src_prefix, dst_prefix):
function test_double_copy (line 130) | def test_double_copy(mscp, src_prefix, dst_prefix, s1, s2, d1, d2):
function test_double_copy_with_ipv6_notation (line 144) | def test_double_copy_with_ipv6_notation(mscp, src_prefix, dst_prefix, s1...
function test_double_copy_with_user_and_ipv6_notation (line 159) | def test_double_copy_with_user_and_ipv6_notation(mscp, src_prefix, dst_p...
function test_dir_copy (line 194) | def test_dir_copy(mscp, src_prefix, dst_prefix, src_dir, dst_dir, src, d...
function test_dir_copy_single (line 216) | def test_dir_copy_single(mscp, src_prefix, dst_prefix, src_dir, dst_dir,...
function test_override_single_file (line 224) | def test_override_single_file(mscp, src_prefix, dst_prefix):
function test_copy_file_under_root_to_dir (line 238) | def test_copy_file_under_root_to_dir(mscp, src_prefix, dst_prefix):
function test_dst_has_suffix_slash (line 248) | def test_dst_has_suffix_slash(mscp, src_prefix, dst_prefix):
function test_remote_path_contains_tilde (line 269) | def test_remote_path_contains_tilde(mscp, src_path, dst_path):
function test_remote_path_contains_tilde2 (line 288) | def test_remote_path_contains_tilde2(mscp):
function test_min_chunk (line 297) | def test_min_chunk(mscp, src_prefix, dst_prefix):
function is_alpine (line 305) | def is_alpine():
function test_glob_src_path (line 330) | def test_glob_src_path(mscp, src_prefix, dst_prefix,
function test_thread_affinity (line 341) | def test_thread_affinity(mscp, src_prefix, dst_prefix):
function test_cannot_override_file_with_dir (line 351) | def test_cannot_override_file_with_dir(mscp, src_prefix, dst_prefix):
function test_transfer_zero_bytes (line 359) | def test_transfer_zero_bytes(mscp, src_prefix, dst_prefix):
function test_override_dst_having_larger_size (line 366) | def test_override_dst_having_larger_size(mscp, src_prefix, dst_prefix):
function test_dont_truncate_dst (line 373) | def test_dont_truncate_dst(mscp, src_prefix, dst_prefix):
function test_copy_readonly_file (line 382) | def test_copy_readonly_file(mscp, src_prefix, dst_prefix):
function test_dont_make_conns_more_than_chunks (line 395) | def test_dont_make_conns_more_than_chunks(mscp, src_prefix, dst_prefix):
function test_bwlimit (line 414) | def test_bwlimit(mscp, src_prefix, dst_prefix):
function test_set_port_ng (line 428) | def test_set_port_ng(mscp, src_prefix, dst_prefix, src, dst):
function test_set_port_ok (line 435) | def test_set_port_ok(mscp, src_prefix, dst_prefix, src, dst):
function test_v4only (line 439) | def test_v4only(mscp):
function test_v6only (line 446) | def test_v6only(mscp):
function test_v4_to_v6_should_fail (line 453) | def test_v4_to_v6_should_fail(mscp):
function test_v6_to_v4_should_fail (line 459) | def test_v6_to_v4_should_fail(mscp):
function test_quiet_mode (line 465) | def test_quiet_mode(capsys, mscp):
function test_set_conn_interval (line 477) | def test_set_conn_interval(mscp, src_prefix, dst_prefix):
function test_compression (line 491) | def test_compression(mscp, src_prefix, dst_prefix, compress):
function test_ccalgo (line 498) | def test_ccalgo(mscp, src_prefix, dst_prefix):
function test_config_ok (line 519) | def test_config_ok(mscp, src_prefix, dst_prefix):
function test_config_ng (line 535) | def test_config_ng(mscp, src_prefix, dst_prefix):
function test_inline_option_ok (line 555) | def test_inline_option_ok(mscp, src_prefix, dst_prefix, option):
function test_inline_option_ng (line 570) | def test_inline_option_ng(mscp, src_prefix, dst_prefix, option):
function test_porxyjump_ok (line 579) | def test_porxyjump_ok(mscp, src_prefix, dst_prefix):
function test_porxyjump_ng (line 591) | def test_porxyjump_ng(mscp, src_prefix, dst_prefix):
function test_specify_passphrase_via_env (line 601) | def test_specify_passphrase_via_env(mscp):
function test_specify_invalid_passphrase_via_env (line 610) | def test_specify_invalid_passphrase_via_env(mscp):
function test_specify_password_via_env (line 618) | def test_specify_password_via_env(mscp):
function test_specify_invalid_password_via_env (line 627) | def test_specify_invalid_password_via_env(mscp):
function move_pubkey_temporally (line 637) | def move_pubkey_temporally():
function test_passwordauth_without_pubkey (line 659) | def test_passwordauth_without_pubkey(move_pubkey_temporally,
function test_10k_files (line 674) | def test_10k_files(mscp, src_prefix, dst_prefix):
function test_checkpoint_dump_and_resume (line 685) | def test_checkpoint_dump_and_resume(mscp, src_prefix, dst_prefix):
function test_checkpoint_interrupt_large_file (line 701) | def test_checkpoint_interrupt_large_file(mscp, timeout, src_prefix, dst_...
function test_checkpoint_interrupt_many_files (line 719) | def test_checkpoint_interrupt_many_files(mscp, timeout, src_prefix, dst_...
FILE: test/util.py
function check_same_md5sum (line 6) | def check_same_md5sum(fa, fb):
class File (line 10) | class File():
method __init__ (line 11) | def __init__(self, path, size = 0, content = "random", perm = 0o664):
method __repr__ (line 19) | def __repr__(self):
method __str__ (line 22) | def __str__(self):
method make (line 25) | def make(self, size = None):
method make_content_zero (line 41) | def make_content_zero(self):
method make_content_random (line 45) | def make_content_random(self):
method cleanup (line 49) | def cleanup(self, preserve_dir = False):
method md5sum (line 59) | def md5sum(self):
Condensed preview — 71 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (351K chars).
[
{
"path": ".clang-format",
"chars": 22310,
"preview": "# SPDX-License-Identifier: GPL-2.0\n#\n# clang-format configuration file. Intended for clang-format >= 11.\n#\n# For more in"
},
{
"path": ".dockerignore",
"chars": 6,
"preview": "build\n"
},
{
"path": ".github/workflows/build-freebsd.yml",
"chars": 662,
"preview": "name: build on FreeBSD\n\non:\n push:\n branches: [ \"main\", \"dev\" ]\n pull_request:\n branches: [ \"main\", \"dev\" ]\n\njob"
},
{
"path": ".github/workflows/build-macos.yml",
"chars": 1726,
"preview": "name: build on macOS\n\non:\n push:\n branches: [ \"main\", \"dev\" ]\n pull_request:\n branches: [ \"main\", \"dev\" ]\n\nenv:\n"
},
{
"path": ".github/workflows/build-ubuntu.yml",
"chars": 1566,
"preview": "name: build on ubuntu\n\non:\n push:\n branches: [ \"main\", \"dev\" ]\n pull_request:\n branches: [ \"main\", \"dev\" ]\n\nenv:"
},
{
"path": ".github/workflows/codeql.yml",
"chars": 3120,
"preview": "# For most projects, this workflow file will not need changing; you simply need\n# to commit it to your repository.\n#\n# Y"
},
{
"path": ".github/workflows/release.yml",
"chars": 886,
"preview": "name: release\n\non:\n push:\n tags:\n - \"v*.*.*\"\n workflow_dispatch:\n\nenv:\n BUILD_TYPE: Release\n\njobs:\n source"
},
{
"path": ".github/workflows/test.yml",
"chars": 1248,
"preview": "name: test\n\non:\n push:\n branches: [ \"main\", \"dev\" ]\n pull_request:\n branches: [ \"main\", \"dev\" ]\n\nenv:\n BUILD_TY"
},
{
"path": ".gitignore",
"chars": 93,
"preview": "build\nhtml\ncompile_commands.json\nCMakeUserPresets.json\n.*.swp\n.cache\n\ninclude/mscp_version.h\n"
},
{
"path": ".gitmodules",
"chars": 102,
"preview": "[submodule \"libssh\"]\n\tpath = libssh\n\turl = https://git.libssh.org/projects/libssh.git\n\tignore = dirty\n"
},
{
"path": "CMakeLists.txt",
"chars": 9070,
"preview": "cmake_minimum_required(VERSION 3.13)\n\nfile (STRINGS \"VERSION\" MSCP_VERSION)\n\nproject(mscp\n\tVERSION ${MSCP_VERSION}\n\tLANG"
},
{
"path": "Dockerfile/README.md",
"chars": 263,
"preview": "\nDockerfiles for building and testing mscp.\n\ncmake provides custom targets to build and test mscp in the containers\nSee "
},
{
"path": "Dockerfile/almalinux-9.3.Dockerfile",
"chars": 1095,
"preview": "FROM almalinux:9.3\n\nARG REQUIREDPKGS\n\n# install pytest, sshd for test, and rpm-build\nRUN set -ex && yum -y install \\\n\t${"
},
{
"path": "Dockerfile/alpine-3.22.Dockerfile",
"chars": 1623,
"preview": "FROM alpine:3.22\n\n# do not use REQUIREDPKGS build argument because\n# this Dockerfile compiles mscp with conan,so we do n"
},
{
"path": "Dockerfile/arch-base.Dockerfile",
"chars": 1242,
"preview": "FROM archlinux:base\n\nARG REQUIREDPKGS\n\n# install pyest and openssh for test\nRUN set -ex && pacman -Syy && pacman --nocon"
},
{
"path": "Dockerfile/build-deb.Dockerfile",
"chars": 415,
"preview": "FROM ubuntu:22.04\n\nARG REQUIREDPKGS\n\nARG DEBIAN_FRONTEND=noninteractive\nRUN set -ex && apt-get update && apt-get install"
},
{
"path": "Dockerfile/build-srpm.Dockerfile",
"chars": 552,
"preview": "FROM rockylinux:9\n\nARG REQUIREDPKGS\nARG MSCP_VERSION\n\n# install pytest, sshd for test, and rpm-build\nRUN set -ex && yum "
},
{
"path": "Dockerfile/rocky-8.9.Dockerfile",
"chars": 1100,
"preview": "FROM rockylinux:8.9\n\nARG REQUIREDPKGS\n\n# install pytest, sshd for test, and rpm-build\nRUN set -ex && yum -y install \\\n\t$"
},
{
"path": "Dockerfile/rocky-9.3.Dockerfile",
"chars": 1100,
"preview": "FROM rockylinux:9.3\n\nARG REQUIREDPKGS\n\n# install pytest, sshd for test, and rpm-build\nRUN set -ex && yum -y install \\\n\t$"
},
{
"path": "Dockerfile/ubuntu-20.04.Dockerfile",
"chars": 1052,
"preview": "FROM ubuntu:20.04\n\nARG REQUIREDPKGS\n\nARG DEBIAN_FRONTEND=noninteractive\nRUN set -ex && apt-get update && apt-get install"
},
{
"path": "Dockerfile/ubuntu-22.04.Dockerfile",
"chars": 985,
"preview": "FROM ubuntu:22.04\n\nARG REQUIREDPKGS\n\nARG DEBIAN_FRONTEND=noninteractive\nRUN set -ex && apt-get update && apt-get install"
},
{
"path": "Dockerfile/ubuntu-24.04.Dockerfile",
"chars": 978,
"preview": "FROM ubuntu:24.04\n\nARG REQUIREDPKGS\n\nARG DEBIAN_FRONTEND=noninteractive\nRUN set -ex && apt-get update && apt-get install"
},
{
"path": "Doxyfile",
"chars": 163,
"preview": "\nPROJECT_NAME\t= libmscp\n\nGENERATE_HTML\t= YES\nGENERATE_CHI\t= NO\nGENERATE_LATEX\t= NO\nGENERATE_RTF\t= NO\nGENERATE_MAN\t= NO\n\n"
},
{
"path": "LICENSE",
"chars": 35149,
"preview": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free "
},
{
"path": "README.md",
"chars": 3340,
"preview": "# mscp: multi-threaded scp\n\n[![build on ubuntu](https://github.com/upa/mscp/actions/workflows/build-ubuntu.yml/badge.svg"
},
{
"path": "VERSION",
"chars": 6,
"preview": "0.2.4\n"
},
{
"path": "conanfile.txt",
"chars": 77,
"preview": "[requires]\nzlib/1.2.11\nopenssl/1.1.1t\n\n[generators]\nCMakeDeps\nCMakeToolchain\n"
},
{
"path": "doc/RELEASE.md",
"chars": 1877,
"preview": "\n\n## Build mscp as deb package\n\n`make build-deb` produces a mscp deb package and related files. This\ntarget builds mscp "
},
{
"path": "doc/mscp.1.in",
"chars": 9574,
"preview": ".TH MSCP 1 \"@MSCP_BUILD_VERSION@\" \"mscp\" \"User Commands\"\n\n.SH NAME\nmscp \\- copy files over multiple SSH connections\n\n.SH"
},
{
"path": "doc/mscp.rst",
"chars": 9438,
"preview": "====\nMSCP\n====\n\n:Date: v0.2.4\n\nNAME\n====\n\nmscp - copy files over multiple SSH connections\n\nSYNOPSIS\n========\n\n**mscp** ["
},
{
"path": "include/config.h.in",
"chars": 437,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _CONFIG_H_\n#define _CONFIG_H_\n\n#define MSCP_VERSION\t\t\"@MSCP_VERSION@"
},
{
"path": "include/mscp.h",
"chars": 8867,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _MSCP_H_\n#define _MSCP_H_\n\n/**\n * @file mscp.h\n *\n * @brief mscp lib"
},
{
"path": "patch/README.md",
"chars": 271,
"preview": "\nPatches in this directory introduce enhancements for libssh including\n`sftp_async_write()` and `sftp_async_write_end()`"
},
{
"path": "patch/libssh-0.10.4.patch",
"chars": 13941,
"preview": "diff --git a/ConfigureChecks.cmake b/ConfigureChecks.cmake\nindex 7103f303..c64eb39d 100644\n--- a/ConfigureChecks.cmake\n+"
},
{
"path": "patch/libssh-0.10.6-2-g6f1b1e76.patch",
"chars": 18750,
"preview": "diff --git a/CMakeLists.txt b/CMakeLists.txt\nindex a64b7708..c6344a5a 100644\n--- a/CMakeLists.txt\n+++ b/CMakeLists.txt\n@"
},
{
"path": "patch/libssh-0.10.6.patch",
"chars": 13932,
"preview": "diff --git a/ConfigureChecks.cmake b/ConfigureChecks.cmake\nindex 9de10225..0f3d20ed 100644\n--- a/ConfigureChecks.cmake\n+"
},
{
"path": "patch/libssh-0.11.2.patch",
"chars": 18682,
"preview": "diff --git a/ConfigureChecks.cmake b/ConfigureChecks.cmake\nindex 8765dc6e..766e7d16 100644\n--- a/ConfigureChecks.cmake\n+"
},
{
"path": "patch/libssh-0.9.6.patch",
"chars": 5238,
"preview": "diff --git a/DefineOptions.cmake b/DefineOptions.cmake\nindex b82a5018..f1f2ab9d 100644\n--- a/DefineOptions.cmake\n+++ b/D"
},
{
"path": "rpm/.gitignore",
"chars": 32,
"preview": "\n# generated by cmake\nmscp.spec\n"
},
{
"path": "rpm/mscp.spec.in",
"chars": 1573,
"preview": "Name:\t\tmscp\nVersion:\t@MSCP_VERSION@\nRelease:\t1%{?dist}\nSummary:\tmscp, fast file transfer over multiple SSH connections\n\n"
},
{
"path": "scripts/install-build-deps.sh",
"chars": 1405,
"preview": "#!/usr/bin/env bash\n#\n# Install build dpenedencies.\n\nset -e\n#set -u\n\nfunction print_help() {\n\techo \"$0 [options]\"\n\techo "
},
{
"path": "scripts/test-in-container.sh",
"chars": 968,
"preview": "#!/bin/bash -e\n#\n# Run this script in mscp docker containers.\n# This script runs end-to-end test with installed mscp.\n\ns"
},
{
"path": "src/atomic.h",
"chars": 1736,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _ATOMIC_H_\n#define _ATOMIC_H_\n\n#include <stdlib.h>\n#include <assert."
},
{
"path": "src/bwlimit.c",
"chars": 2436,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <errno.h>\n\n#include <bwlimit.h>\n#include <platform.h>\n\n#define time"
},
{
"path": "src/bwlimit.h",
"chars": 762,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _BWLIMIT_H_\n#define _BWLIMIT_H_\n\n#include <stdbool.h>\n#include <stdi"
},
{
"path": "src/checkpoint.c",
"chars": 13949,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <fcntl.h>\n#include <sys/uio.h>\n#include <arpa/inet.h>\n\n#include <pa"
},
{
"path": "src/checkpoint.h",
"chars": 760,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _CHECKPOINT_H_\n#define _CHECKPOINT_H_\n\n#include <pool.h>\n\n/* checkpo"
},
{
"path": "src/fileops.c",
"chars": 7971,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <stdlib.h>\n#include <string.h>\n#include <errno.h>\n#include <fcntl.h"
},
{
"path": "src/fileops.h",
"chars": 1575,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <dirent.h>\n#include <sys/stat.h>\n#include <glob.h>\n\n#include <ssh.h"
},
{
"path": "src/main.c",
"chars": 18647,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <stdio.h>\n#include <stdlib.h>\n#include <stdarg.h>\n#include <fcntl.h"
},
{
"path": "src/minmax.h",
"chars": 196,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _MINMAX_H_\n#define _MINMAX_H_\n\n#define min(a, b) (((a) > (b)) ? (b) "
},
{
"path": "src/mscp.c",
"chars": 16901,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <stdbool.h>\n#include <unistd.h>\n#include <math.h>\n#include <pthread"
},
{
"path": "src/openbsd-compat/openbsd-compat.h",
"chars": 191,
"preview": "#ifndef _OPENBSD_COMPAT_H\n#define _OPENBSD_COMPAT_H\n\n#include \"config.h\"\n\n#ifndef HAVE_STRLCAT\nsize_t strlcat(char *dst,"
},
{
"path": "src/openbsd-compat/strlcat.c",
"chars": 1804,
"preview": "/*\t$OpenBSD: strlcat.c,v 1.13 2005/08/08 08:05:37 espie Exp $\t*/\n\n/*\n * Copyright (c) 1998 Todd C. Miller <Todd.Miller@c"
},
{
"path": "src/path.c",
"chars": 13215,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <string.h>\n#include <unistd.h>\n#include <dirent.h>\n#include <sys/st"
},
{
"path": "src/path.h",
"chars": 1776,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _PATH_H_\n#define _PATH_H_\n\n#include <limits.h>\n#include <fcntl.h>\n#i"
},
{
"path": "src/platform.c",
"chars": 2948,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifdef __APPLE__\n#include <stdlib.h>\n#include <sys/types.h>\n#include <sys/ti"
},
{
"path": "src/platform.h",
"chars": 1277,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _PLATFORM_H_\n#define _PLATFORM_H_\n\n#include <config.h>\n\n#include <pt"
},
{
"path": "src/pool.c",
"chars": 1821,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <string.h>\n#include <stdlib.h>\n#include <pool.h>\n\n#define DEFAULT_S"
},
{
"path": "src/pool.h",
"chars": 2725,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _POOL_H_\n#define _POOL_H_\n\n#include <stdbool.h>\n#include <stddef.h>\n"
},
{
"path": "src/print.c",
"chars": 364,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n\n#include <print.h>\n\n/* message print functions */\nstatic int __print_severi"
},
{
"path": "src/print.h",
"chars": 1145,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _PRINT_H_\n#define _PRINT_H_\n\n#include <libgen.h>\n#include <stdio.h>\n"
},
{
"path": "src/ssh.c",
"chars": 10562,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#include <stdio.h>\n#include <string.h>\n#include <unistd.h>\n#include <stdlib."
},
{
"path": "src/ssh.h",
"chars": 540,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _SSH_H_\n#define _SSH_H_\n\n#include <stdbool.h>\n#include \"libssh/libss"
},
{
"path": "src/strerrno.c",
"chars": 839,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n\n#include <errno.h>\n#include <stdio.h>\n#include <string.h>\n#include <stdarg."
},
{
"path": "src/strerrno.h",
"chars": 1066,
"preview": "/* SPDX-License-Identifier: GPL-3.0-only */\n#ifndef _STRERRNO_\n#define _STRERRNO_\n\n#include <libgen.h> /* basename() */\n"
},
{
"path": "test/.gitignore",
"chars": 12,
"preview": "__pycache__\n"
},
{
"path": "test/README.md",
"chars": 238,
"preview": "\nThis test assumes that the user executing the test can ssh to the\nlocalhost without password.\n\n- Run pytest through cte"
},
{
"path": "test/conftest.py",
"chars": 237,
"preview": "import pytest\n\ndef pytest_addoption(parser):\n parser.addoption(\"--mscp-path\", default = \"mscp\",\n "
},
{
"path": "test/test_e2e.py",
"chars": 25994,
"preview": "\n\"\"\"\ntest_e2e.py: End-to-End test for mscp executable.\n\"\"\"\n\nimport platform\nimport pytest\nimport getpass\nimport datetime"
},
{
"path": "test/util.py",
"chars": 1793,
"preview": "\nimport hashlib\nimport os\n\n\ndef check_same_md5sum(fa, fb):\n return (fa.md5sum() == fb.md5sum())\n\n\nclass File():\n d"
}
]
About this extraction
This page contains the full source code of the upa/mscp GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 71 files (320.7 KB), approximately 96.1k tokens, and a symbol index with 280 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.