Best Ways to Edit Files over SSH (Linux): sed vs Perl and All the Better Options Print

Summary (TL;DR) ⚡

• Simple, line‑based tweaks → sed -i.bak
• Advanced regex / multi‑line / logic → perl -pi -e
• Structured configs → jq (JSON), yq (YAML), crudini (INI), xmlstarlet (XML)
• Bulk & repeatable changes → version control + scripts or Ansible
• Always dry‑run, back up, and scope edits ✅

🧭 Table of Contents

1. 🔒 Safety Rules (Do This Every Time)

2. 🆚 sed vs Perl: When to Choose Which

3. 🧰 Essential Alternatives (Beyond sed/Perl)

4. 🍳 The Cookbook: Common Editing Tasks

5. 🧪 Dry‑Run Patterns & Safe Rollback

6. 🧱 Structured Config Editing (JSON/YAML/INI/XML)

7. 🗂️ Bulk, Cross‑File & Cross‑Server Operations

8. 🧠 Tips for Editors (vim/nvim, nano, ed/ex)

9. 🧯 Troubleshooting & Pitfalls

10. ✅ Final Checklist (Print & Stick on the Wall)

1) 🔒 Safety Rules (Do This Every Time)

• Preview first (no in‑place changes):

  sed 's/OLD/NEW/g' file | diff -u file -
  perl -pe 's/OLD/NEW/g' file | diff -u file -
  

• Create a backup you can later clean:

  sed -i.bak 's/OLD/NEW/g' file
  perl -pi.bak -e 's/OLD/NEW/g' file
  

• Scope precisely to only the intended files:

  git ls-files '*.conf' | xargs -r sed -i.bak 's/OLD/NEW/g'
  find config -type f -name '*.ini' -print0 | xargs -0 -r perl -pi.bak -e 's/OLD/NEW/g'
  

• Fail fast in scripts:

  set -euo pipefail
  

• Privileged files: write safely via sudo tee to avoid partial writes:

  sed 's/OLD/NEW/' file | sudo tee file >/dev/null
  

• Atomic updates for critical configs: write to temp → mv into place.

2) 🆚 sed vs Perl: When to Choose Which

Rules of thumb:

• Pick sed for common, line‑based edits and portability on Linux servers.

• Pick Perl when your regex needs teeth (look‑arounds, multi‑line stanzas, conditional logic).

3) 🧰 Essential Alternatives (Beyond sed/Perl)

Use the right tool for the data. Text is not always “just text”.

• awk 🐣 – Great for columnar transforms, selecting fields, and conditional edits.

• vim/nvim 🧑‍💻 – Ex‑mode & macros for complex, repeatable edits across buffers/files.

• nano ✏️ – Quick manual fix in a pinch; easy for non‑vim users.

• ed/ex 📜 – Scriptable line editors; perfect for automation in minimal environments.

• ripgrep (rg) + sd 🚀 – Ultra‑fast search + ergonomic replace (sd is a friendlier sed for many cases).

• jq / yq / crudini / xmlstarlet 🧩 – Edit JSON/YAML/INI/XML structurally (no fragile regex).

• Augeas (augtool) 🪞 – Policy‑driven config editing using lenses (safer for many system configs).

• Templating 🧱 – Generate configs instead of patching them: envsubst, gomplate, jinja2-cli.

• Orchestration 🧭 – Idempotent, documented edits at scale: Ansible (lineinfile, blockinfile, template).

• moreutils’s sponge 🧽 – Avoid truncation when reading & writing the same file: cmd | sponge file.

4) 🍳 The Cookbook: Common Editing Tasks

4.1 🔁 Global Replace (with safe delimiter)

sed -i.bak 's|/var/www/app|/srv/app|g' /etc/app.conf
perl -pi.bak -e 's|/var/www/app|/srv/app|g' /etc/app.conf

4.2 ➕ Append a Line After a Match

sed -i.bak '/^\[server\]/a max_connections = 200' /etc/app.ini
perl -pi.bak -e 's/^\[server\]\n/[server]\nmax_connections = 200\n/' /etc/app.ini

4.3 🧽 Delete Lines Matching a Pattern

sed -i.bak '/^#\s*DEBUG/d' /etc/app.conf
perl -ni.bak -e 'print unless /^#\s*DEBUG/' /etc/app.conf

4.4 🗣️ Comment / Uncomment a Directive

# Comment
sed -i.bak 's/^(Listen 8080)/# \1/' /etc/app.conf
perl -pi.bak -e 's/^(\s*Listen 8080)/# $1/' /etc/app.conf

# Uncomment
sed -i.bak 's/^#\s*(Listen 8080)/\1/' /etc/app.conf
perl -pi.bak -e 's/^#\s*(Listen 8080)/$1/' /etc/app.conf

4.5 🎛️ Change a Key=Value Safely (Idempotent)

sed -i.bak -E 's|^(\s*timeout\s*=\s*).*|\1"60s"|' /etc/app.conf
perl -pi.bak -e 's/^(\s*timeout\s*=\s*).*/${1}"60s"/' /etc/app.conf

4.6 📚 Within a Section Only (Bounded Range)

# sed range between headings
sed -i.bak '/^\[server\]/,/^\[/{s/^port=.*/port=9090/}' /etc/app.ini

# Perl stateful section tracking
perl -pi.bak -e '$in=1 if /^\[server\]/; $in=0 if /^\[.*\]/ && !/^\[server\]/; $in && s/^port=.*/port=9090/' /etc/app.ini

4.7 🧵 Multi‑Line Insertion (Prefer Perl)

perl -0777 -pe 's/(\[logging\]\n)/$1level = "info"\nformat = "json"\n/s' -i.bak /etc/app.ini

4.8 🔢 Version Bump (Minor)

perl -pi.bak -e 's/^version\s*=\s*(\d+)\.(\d+)/"version=".( $1 ).".".($2+1)/e' app.ini

4.9 🧹 CRLF Cleanup (Before Regex Edits)

sed -i.bak 's/\r$//' file
perl -pi.bak -e 's/\r$//' file

4.10 🧮 Column Edits with awk

# Increase 3rd column by 10
awk '{ $3 += 10 }1' input.tsv > out.tsv

5) 🧪 Dry‑Run Patterns & Safe Rollback

• Preview changes with diff:

  perl -pe 's/foo/bar/g' file | diff -u file - | sed -n '1,200p'
  

• Git‑aware bulk edits:

  git ls-files -z '*.conf' | xargs -0 -r perl -pi.bak -e 's/OLD/NEW/g'
  git diff --name-only | xargs -r -n1 sed -n '1,50p'
  

• Rollback quickly:

  git restore --source=HEAD -- :/
  # or restore a single file from its backup
  cp file.bak file
  

• Cleanup backups after validation:

  find . -type f -name '*.bak' -delete
  

6) 🧱 Structured Config Editing (JSON/YAML/INI/XML)

Regex isn’t a parser. Prefer structure‑aware tools.

• JSON → jq

  jq '.logging.level="info"' config.json | sponge config.json
  

• YAML → yq

  yq -i '.server.port = 9090' config.yaml
  

• INI → crudini

  crudini --set /etc/app.ini server port 9090
  

• XML → xmlstarlet

  xmlstarlet ed -u '/config/server/port' -v 9090 config.xml > config.xml.tmp && mv config.xml.tmp config.xml
  

• Augeas (augtool) – Lens‑based, safe edits for many system files:

  augtool set /files/etc/app.ini/server/port 9090
  augtool save
  

7) 🗂️ Bulk, Cross‑File & Cross‑Server Operations

• Find only the targets (speed + safety):

  rg -l --hidden --glob '!vendor/*' 'pattern' | xargs -r perl -pi.bak -e 's/OLD/NEW/g'
  

• Parallelize for speed (carefully):

  rg -l 'OLD' | parallel -j4 perl -pi.bak -e 's/OLD/NEW/g' {}
  

• Idempotent at scale: Ansible
  Use modules: lineinfile, blockinfile, replace, template. Example:

  - name: Ensure timeout is 60s
    ansible.builtin.lineinfile:
      path: /etc/app.conf
      regexp: '^\s*timeout\s*='
      line: 'timeout = "60s"'
      backup: yes
  

• Version everything with Git or Etckeeper for /etc.

8) 🧠 Tips for Editors (vim/nvim, nano, ed/ex)

• vim Ex‑mode one‑shot:

  vim -Es +'g/^#\s*DEBUG/d' +wq /etc/app.conf
  

• Across many files:

  vim -p $(git ls-files '*.conf') \
    +'argdo %s/OLD/NEW/g | update' +qa
  

• Macros & visual block: Perfect for column edits or repeated transformations.

• nano: Easy, minimal; enable line numbers and soft wrap in ~/.nanorc.

• ed/ex: Deterministic, scriptable in minimal shells/containers.

9) 🧯 Troubleshooting & Pitfalls

• Nothing changed? Your regex didn’t match. Verify:

  rg -n 'your-regex' file
  

• Over‑matching: Anchor with ^/$, or use word boundaries (\b in Perl).

• Delimiter hell: Switch delimiters: s|/a/b|/x/y|g.

• BSD vs GNU sed -i: On macOS, -i'' means “no backup”; on Linux GNU sed, -i.bak is standard.

• Locales: For byte‑wise speed & predictability: LC_ALL=C.

• Binary or huge files: Don’t sed/perl binaries. Stream or split large files if memory is tight.

10) ✅ Final Checklist (Print & Stick on the Wall)

• Preview with diff

• Back up (-i.bak)

• Scope the target set (git ls-files/find/rg)

• Choose the right tool (sed/Perl/jq/yq/crudini/xmlstarlet)

• Apply atomically for critical configs

• Commit to version control & document the change

🧩 Copy‑Ready Bulk‑Edit Template (Safe & Git‑Aware)

#!/usr/bin/env bash
set -euo pipefail

PATTERN='^(\s*timeout\s*=\s*).*$'
REPLACEMENT='${1}"60s"'

# Scope: only tracked INI files under config/
git ls-files -z 'config/**/*.ini' \
| xargs -0 -r perl -pi.bak -e "s/${PATTERN}/${REPLACEMENT}/"

echo "Changes staged:"; git status --porcelain

echo "Preview diff (first 200 lines):"
git diff | sed -n '1,200p'

# Rollback (if needed):  git restore --source=HEAD -- :/
# Cleanup backups when satisfied:
# find config -type f -name '*.bak' -delete

📎 Handy One‑Liners Recap

• Replace path everywhere (safe delimiter):

  perl -pi.bak -e 's|/var/www|/srv/www|g' $(git ls-files '*.conf')
  

• Insert block after header:

  perl -0777 -pe 's/(\[logging\]\n)/$1level="info"\nformat="json"\n/s' -i.bak /etc/app.ini
  

• Structured JSON tweak:

  jq '.service.enabled=true' config.json | sponge config.json
  

• YAML port update:

  yq -i '.server.port=8081' config.yaml
  

• INI using crudini:

  crudini --set /etc/app.ini server port 8081
  

💡 Pro Tip: For customer‑facing SOPs, turn these into parameterized scripts or Ansible roles. It’s safer, faster, and easier to audit.

📚 Further help for your team & clients:
Knowledgebase: https://www.domainindia.com/knowledgebase
Submit a Ticket: https://www.domainindia.com/support