Skip to content

Linear time list operations#10

Open
haampie wants to merge 3 commits into
mainfrom
hs/fix/speed
Open

Linear time list operations#10
haampie wants to merge 3 commits into
mainfrom
hs/fix/speed

Conversation

@haampie
Copy link
Copy Markdown
Member

@haampie haampie commented May 14, 2026

Avoids string concatenation overhead in extend and preextend

"tricks" used:

  • Build a local temporary string instead of extending a big string one item at a time (quadratic complexity)
  • When prefix not set: use set -- and "$*" with different IFS for fast concatenation, presumably linear complexity.
  • When prefix is set: shift the first element, followed by a tight loop (still quadratic complexity).
  • Eliminate the array reversal in preextend

In the included benchmark, runtime goes from 100ms to 21ms on my macbook.

haampie added 2 commits May 15, 2026 11:11
Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Copy link
Copy Markdown

@sethrj sethrj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! Good testing and clever scripting.

Comment thread cc.sh
Comment on lines +169 to +180
setsep "$_dst"; _dst_sep="$sep"

if [ -z "$_prefix" ]; then
# Fast concatenation when no prefix is needed
IFS="$_dst_sep"; _ext_str="$*"; unset IFS
else
_ext_str="${_prefix}$1"
shift
for elt; do
_ext_str="${_ext_str}${_dst_sep}${_prefix}${elt}"
done
fi
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a nontrivial performance penalty for merging these and the similar lines at 135 into a single function, e.g. one that uses printf and is called with _ext_str="$(merge "$_dst" "$sep" "$_prefix" "$@")"? Hiding the symbol-filled code (or at least making clear that it's an exact duplicate) woudl be good.

Copy link
Copy Markdown
Member Author

@haampie haampie May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is a bit dense and repetitive... So far we've tried to avoid printf, or rather subshells "$(printf ...)". I'm not sure if it can be improved without subshells.

Comment thread cc.sh
Copy link
Copy Markdown
Collaborator

@kwryankrattiger kwryankrattiger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, the expected/result are consistent for all of the tests look correct.

I ran all of the test and the benchmark before and after these changes and found one bug in the old preextend command that was double prepending the PREFIX. The speed up for the benchmark as about 4x which is pretty good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants