Working on a script to automate a few things. Been having and issue with running a sed command to inser the output of curl to a line number in a file
I have tried multiple variations of the command and i get nothing but errors from sed.
sed "49i/.*/\"$(curl URL)\"/" file.php
sed: -e expression #1, char 105: extra characters after commandAny ideas appericated Thanks
12 Answers
The output of curl URL (after $() removes trailing newline(s)) probably contains newline character(s). It gets inserted in the command line argument that is supposed to be your script for sed. The tool parses the code, encounters a newline after 49i and interprets what follows as the next command in the script.
You got an error. It could be worse: the output from curl could inject syntactically valid sed code into the script. You don't want this.
Even if you use $(curl URL | tr -d '\n') to remove all newlines, the output of curl may still not be inserted verbatim (e.g. backslashes will have special meaning).
Solution: let curl write to a temporary file, then let sed print the file. The content of the file will not be interpreted as code.
It seems you want the output from curl to appear before the original content of the 49th line. If so, you need to use r on the 48th line (because r schedules text for later output).
Your code tries to insert something in a form /.*/"curl output here"/. If this is not an artifact from trying to use the syntax of s with i, then you should write these additional fragments to the file as well.
Normally you can make curl write to a file by using curl -o. In this case it may not be what you want because it doesn't remove trailing newlines (while $() you used does). So maybe this:
tmpfile="$(mktemp)"
printf '/.*/"%s"/' "$(curl URL)" >"$tmpfile"
sed "48 r $tmpfile" file.php
rm "$tmpfile"Note in general it will fail if the output from curl is too long. A more robust solution is to make curl print to the file directly and to remove the newline(s) from the end in the next step.
The path to the temporary file must not contain newlines. If it does contain newlines then some part(s) of the path will be interpreted as sed command(s), possibly invalid; a bug somewhat similar to what you experienced. The path won't contain newlines, unless your $TMPDIR contains them (but it really shouldn't). In case of problems (e.g. mktemp is unavailable) use another method of picking a temporary location. In the most basic setup the location may be hardcoded (but then you shouldn't run more than one instance of the script simultaneously).
sed's i...\n command doesn't have the same syntax as s/.../.../..., which I think is the main issue here. Along with the fact that your curl output likely contains newlines, which end up terminating the i command.
s/find/replace/ will search for find and swap it for replace
In the example below, the text that matches the regex 3 is swapped out.
$ echo -e "1\n2\n3\n4\n5" | sed 's/3/new line/'
1
2
new line
4
5inew line will insert new line before the matched line(s)
In the example below, we've not given an address or match, so every input line has new line inserted before it.
$ echo -e "1\n2\n3\n4\n5" | sed 'inew line'
new line
1
new line
2
new line
3
new line
4
new line
5What you're trying to do appears to be insert the new line at a given line number (aka "address"), which can be achieved like this. Note that the i command is terminated by a newline (or end of string) - the new line is inserted before, it does not replace line three.
$ echo -e "1\n2\n3\n4\n5" | sed '3inew line'
1
2
new line
3
4
5Now, as you're using the output of curl, you may well find that this contains a newline in it, which would terminate the i command... so you'll need to convert curl's output into a single line, or only include the first line - you can use tools like head or xxd to help:
$ curl "${url}"
line one
line two
line threeOnly keep first line:
$ new_line="$(curl -s "${url}" | head -n 1)"
$ echo -e "1\n2\n3\n4\n5" | sed '3i'"${new_line}"
1
2
line one
3
4
5Escape all line breaks (\r\n and \n) into the literal characters \n. sed will then convert these back into a line break:
$ new_line="$(curl -s "${url}" | xxd -p | sed -re 's/(0d)?0a/5c6e/g;s/5c6e$//' | xxd -r -p)"
$ echo -e "1\n2\n3\n4\n5" | sed '3i'"${new_line}"
1
2
line one
line two
line three
3
4
5Escape all line breaks (\r\n and \n) into the literal characters \\n. sed will then convert these into a the literal characters \n, resulting in a single line being inserted:
$ new_line="$(curl -s "${url}" | xxd -p | sed -re 's/(0d)?0a/5c5c6e/g;s/5c5c6e$//' | xxd -r -p)"
$ echo -e "1\n2\n3\n4\n5" | sed '3i'"${new_line}"
1
2
line one\nline two\nline three
3
4
5This demonstrates that it is possible, however a better approach may be to just reconstruct the file using head, curl and tail, like this... it'll probably be more readable in the future:
$ echo -e "1\n2\n3\n4\n5" > "${file}"
$ tmpfile="$(mktemp)"
$ ( head -n 2 < "${file}"; curl "${url}"; tail -n +3 < "${file}" ) > "${tmpfile}"
$ mv "${tmpfile}" "${file}"
$ cat "${file}"
1
2
line one
line two
line three
3
4
5
