Vim's Pipes

A Little Drop of Prudence

I like my Vim with a little drop of prudence

I frequently create temporary macros or maps for ad hoc edits when I find myself having to do the same job more than a few times over. This is a good thing to spend some time reflecting on in your editing. If you’re in the heat of the moment and don’t want to break concentration or waste time on R&D right now, make a note to come back when you have time to look at your current editing inefficiency. I recommend setting up a practise file (something I mention in learnvim) which you can quickly jump to using a global bookmark.

Setting up a Practise File :e ~/vim-practise.txt mP This sets a global mark that can be jumped to from anywhere within vim using the normal mode ' command. (:help 'A) Jumping to your Practise File 'P

So, there you are editing away on another dreary Tuesday and in a moment of lucidity you realise you’ve just mashed the same key pattern a dozen times over — you’ve just discovered an inefficiency! Awesome.

Quick check: “Do I have time to investigate and optimise this now?”

No: :-( Sucks to be you. Quick! To the Practise File! Make a quick note about this so that you can come back to it on your morning tea break.

Yes: :-) You soldier! Yank a sample snippet of the problem at hand and then… Quick! To the Practise File! Paste in your snippet and start experimenting with ways to optimise the necessary changes. Is there a map or macro you can make to wrap these steps up into a fast and simple solution? Take your macro/map back to the real work you were doing before this R&D diversion and finish the rest of those lame edits with the genuine vim you should be applying to life.

“But couldn’t I have just experimented in my original file?”

Sure… but then you lose the problem. You’re left with just a finished solution. A useless page of sterile, problemless text. That might please your boss and clients, but it’s just no good for your continued development as a vimmer. You’ll grow more by squirreling away interesting little nasties like this that you find in the wild so that you can revisit them during quieter moments as part of your Deliberate Practise regimen.

Genetic Algorithms in VimL (Part I)

Burak Kanber is into machine learning. I was entertained by his Hello World genetic algorithm example and, in alignment with his implementation language agnosticism, I thought I'd write a version in VimL:

This depends on vim-rng for the random number stuff.

let s:Chromosome = {}

function! s:Chromosome.New(...)
  let chromosome = copy(self)
  let chromosome.code = ''
  let chromosome.cost = 9999
  let chromosome.pivot = 0

  if a:0
    let chromosome.code = a:1
    let chromosome.pivot = (strchars(chromosome.code) / 2) - 1
  endif

  return chromosome
endfunction

function! s:Chromosome.random(length)
  let self.code = RandomString(a:length)
  let self.pivot = (a:length / 2) - 1
  return self
endfunction

function! s:Chromosome.mutate(chance)
  if (RandomNumber(100) / 100.0) < a:chance
    let index = RandomNumber(1, strchars(self.code)) - 1
    let upOrDown = RandomNumber(100) <= 50 ? -1 : 1
    let exploded = split(self.code, '\zs')
    let change = nr2char(char2nr(exploded[index]) + upOrDown)
    if index == 0
      let self.code = change . join(exploded[index+1:], '')
    else
      let self.code = join(exploded[0:index-1], '') . change . join(exploded[index+1:], '')
    endif
  endif
  return self
endfunction

function! s:Chromosome.mate(chromosome)
  let child1 = strpart(self.code, 0, self.pivot) . strpart(a:chromosome.code, self.pivot)
  let child2 = strpart(a:chromosome.code, 0, self.pivot) . strpart(self.code, self.pivot)
  return [s:Chromosome.New(child1), s:Chromosome.New(child2)]
endfunction

function! s:Chromosome.calcCost(compareTo)
  let total = 0
  let i = 0
  while i < strchars(self.code)
    let diff = char2nr(self.code[i]) - char2nr(a:compareTo[i])
    let total += diff * diff
    let i += 1
  endwhile
  let self.cost = total
  return self
endfunction

function! s:Chromosome.to_s()
  return self.code . ' (' . string(self.cost) . ')'
endfunction


let s:Population = {}

function! s:Population.New(goal, size)
  let population = copy(self)
  let population.members = []
  let population.goal = a:goal
  let population.generationNumber = 0
  let population.solved = 0

  let size = a:size
  let length = strchars(population.goal)
  while size > 0
    let chromosome = s:Chromosome.New()
    call chromosome.random(length)
    call add(population.members, chromosome)
    let size -= 1
  endwhile

  return population
endfunction

function! s:Population.display()
  % delete
  call setline(1, "Generation: " . self.generationNumber)
  call setline(2, map(copy(self.members), 'v:val.to_s()'))
  redraw
  return self
endfunction

function! s:Population.costly(a, b)
  return float2nr(a:a.cost - a:b.cost)
endfunction

function! s:Population.sort()
  call sort(self.members, self.costly, self)
endfunction

function! s:Population.generation()
  call map(self.members, 'v:val.calcCost(self.goal)')

  call self.sort()
  call self.display()

  let children = self.members[0].mate(self.members[1])
  let self.members = extend(self.members[0:-3], children)

  let i = 0
  while i < len(self.members)
    call self.members[i].mutate(0.5)
    call self.members[i].calcCost(self.goal)
    if self.members[i].code == self.goal
      call self.sort()
      call self.display()
      let self.solved = 1
      break
    endif
    let i += 1
  endwhile

  let self.generationNumber += 1

  return self
endfunction

enew
let population = s:Population.New('Hello, world!', 20)
while population.solved != 1
  call population.generation()
endwhile

To see it run save it in a file and type   :so %   from within vim.

Why, dear bairui, you ask? Well, at this stage... I don't know. It just looked like fun. However, a couple of wild thoughts occurred to me: finding the ideal (good enough; as in 'correct' enough) combination of various vim options to achieve a desired look and behaviour. Take for example the various C indenting styles - what mad combination of &cinoptions, &cinkeys and &cinwords would you need to achieve Frankenstein's Indentation Style? What about getting &formatlistpat right for your preferred markup style? Sure, these might be totally hair-brained ideas -- but they might give you an idea for something less hairy and actually useful. Either way, I plan to keep playing with Burak's tutorial as he progresses through it. Thanks, Burak! :-)

A PEG Parser Generator for Vim

Barry Arthur
v1.2, August 15, 2012
v1.1, October 10, 2011

What is VimPEG?

VimPEG is a Parser Generator which uses the newer Parsing Expression Grammar formalism to specify parse rules.

Why VimPEG?

Vim is a powerful editor. It has lots of features baked right in to make it an editor most awesome. It has a deliciously potent regular expression engine, jaw-dropping text-object manipulations, and fabulous scriptability -- just to name a few of its aces.

One thing our little Vim still lacks, though, is an actual parser. Regular expressions will only get you so far when you're trying to analyse and understand complex chunks of text. If your text is inherently infinite or recursive, then regular expressions become at best combersome, and at worst, useless.

So, Vim needs a parser. I've needed one myself several times when wanting to build a new plugin:

Awesome! This idea will so rock! Now all I need to do is parse <SomeLanguage> and I'll be able to... awww... :-(

I've seen people ask on #vim: How can I <DoSomethingThatNeedsAParser>? And invariably the answer is: You can't. Not easily, anyway. Vimscript is a capable enough language to write your own parser in, but a little alien to most to do so.

You could also use one of the many language bindings that Vim comes bundled with these days to use a parser library in your favourite scripting language. The problem being that your code will only then run on a similarly compiled Vim (not everyone enables these extra language bindings) and with your parser library dependencies.

Beyond those two options, the world of parsing in Vim is quite scant. There exist a small handful of purpose-built recursive descent parsers that target a specific task (like parsing json), but for the general case -- a parser-generator -- you're out of luck. Until now. VimPEG aims to solve this problem.

VimPEG aims to be a 100% VimL solution to your parsing needs.

What would I use VimPEG for?

• You've come to that paralysing sinkhole in your Vimming when you've said to yourself, "Damn... I wish Vim had a parser."
• You've asked for something on #vim and the reply is "you can't do that because Vim doesn't have a parser."
• You're up to your neck in recklessly recursive regexes.

Some ideas:

• An expression calculator (the beginnings of which we explore here.)
• Expanding tokens in typed text (think: snippets, abbrevs, maps.)
• Semantic analysis of code -- for refactoring, reindenting (but sadly not syntax highlighting yet.)
• C Code bifurcation based on #define values -- want to see what the code would look like with #define DEBUG disabled?
• Coffeescript for Vim -- sugar-coating some of the uglies in VimL -- this example will be presented in a subsequent VimPEG article.

In fact, most of these ideas have been explored in part inside the examples/ directory of the VimPEG plugin.

For the purposes of introducing VimPEG and parsing in general (if you're new to it), let's consider a fairly easy example of reading and understanding (perhaps calculating) a sum series of integers. They look like this:

    1 + 2 + 12 + 34

NOTE: Vim can already do this for you, so writing a parser for it here is purely pedagogical -- it's a simple enough example without being utterly devoid of educational value. I hope.

The list can be any (reasonable) length, from a single integer upwards. So, this is a valid input to our parser:

    123

As are all of the following:

    1 + 2
    3 + 4 + 5
    123 + 456 + 789

Stop. Right now. And think: How would you parse such an arbitrarily long series of integers separated by + operators? What tool would you reach for? What if you had to do it in Vim? And :echo eval('1 + 2 + 3') is cheating. :-p

We'll continue to use this example throughout this article and eventually show you how VimPEG solves this little parsing requirement.

But first, let's make sure we're all on the same page about the question: What is parsing?

Parsing

Feel free to skip to the next section if you're comfortable with the following concepts:

• parsing
• pasrer generators
• (E)BNF and PEGs

Let's begin by defining some terms:

What is 'Parsing'?

Parsing is making sense of something. When we want a computer to understand something we've written down for it to do, it needs to 'parse' that writing.  Without going into too much detail yet, let's consider a sentence uttered at one time or another by your parental unit: "Take the rubbish out!". When you (eventually -- after you unplug your iPod, put down your PS3 controller, pocket your smart-phone and wipe the disdain off your face) parse this sentence, your brain goes through two processes:

firstly, syntax recognition:

• it scans the words to make sure they're legitimate:
• they're in a language you know
• they're all valid words, and
• they're all in the right order

and secondly, semantic analysis:

• it filters out the 'meaning' and presents that to a higher actor for further deliberation

In this case, the parser would extract the verb phrase 'take out' and the noun 'rubbish'. Your higher self (sarcasm aside) knows where this magic 'out' place is. We'll come back to these two processes ('syntax recognition' and 'semantic analysis') later.

In the case of our sum series of integers, syntax recognition would involve collecting the sequence of digits that comprise an integer, skipping unnecessary whitespace and expecting either an end of input or a + character and another integer and... so on. If the input contained an alphabetic character it would fail in this phase -- alphabetic characters are just not expected in the input. If the lexical recogniser found two integers separated by whitespace or two + characters in a row...  it would not fail in this phase -- these are all valid tokens in 'this' lexical recogniser.

I am describing the more general process of lexical recognition and it being a separate stage to semantic analysis which is typical of a lot of parsers. PEG parsers, however, do not have separate phases as described here -- they are quite strict about not only what shape the next token must have, but also its purpose in this place (context) of the input. Having two consecutive integers or two consecutive + characters will upset a PEG parser expecting a sum series of integers -- it's just that it gets upset all in its single parse phase.

The semantic analysis phase is all about doing something
"meaningful" with the collected integers. Maybe we should sum them? Maybe we just want to pass back a nested list structure representing the parse tree, like this:

    [1, '+', [2, '+', [3, '+', 4]]]

given this input:

    1 + 2 + 3 + 4

Either way, whatever is done, it's the job of the semantic analysis phase to do so. In our example in this article, we produce a sum of the collected integer series. So, our parser would return: 10 for the example input given above.

What is a 'Parser Generator'?

Writing a parser is not easy. Well, it's not simple. It's fussy. It's messy. There's a lot of repetition and many edge cases and minutia that bores a good coder to tears. Sure, writing your first recursive descent parser is better than sex, but writing your second one isn't. Writing many is as much fun as abstinence. Enough said.

So, we (as fun loving coders) want a better alternative. Parser generators provide that alternative. They generate parsers; which means they do all the boring, tedious, repetitive hard-labour and clerical book-keeping stuff for us. I hope I've painted that with just the right amount of negative emotion to convince you on a subliminal level that Parser Generators are a Good Thing(TM).

How do they generate a parser? or What's a 'PEG'?

Parser Generators are told what to expect (what is valid or invalid) through a grammar -- a set of rules describing the allowed constructs in the language it's reading. Defining these rules in a declarative form is much easier, quicker and less error-prone than hand-coding the equivalent parser.

Bryan Ford recently (circa 2004) described a better way to declare these rules in the form of what he called Parsing Expression Grammars -- PEGs.

NOTE: We used to declare these parsing rules in EBNF, intended for a recursive descent parser (or an LL or LALR or other parser). And before you drown me in comments of "They so still use that, dude!" -- I know. They do.

In a nutshell, PEGs describe what is expected in the input, rather than the (E)BNF approach of describing what is possible. The difference is subtle but liberating. We'll not go too much into that now -- except to say: PEGs offer a cleaner way to describe languages that computers are expected to parse. If you want to re-program your 13 year old brother, you might not reach for a PEG parser generator, but as we're dabbling here in the confines of computers and the valley of vim, PEGs will do just fine.

A major benefit to PEG parsers is that there is no separate lexical analysis phase necessary. Because PEG parsers 'expect' to see the input in a certain way, they can ask for it in those expected chunks. If it matches, great, move on. If it doesn't match, try another alternative. If all the alternatives fail, then the input doesn't match. Allow for backtracking, and you have all you need to parse 'expected' input.

NOTE: VimPEG is not a memoising (packrat) parser -- not yet, anyway.

A brief overview of the PEG parsing rule syntax

• Terminal symbols are concrete and represent actual strings (or in the case of VimPEG, Vim regular expressions) to be matched.
• Non-terminal symbols are names referring to combinations of other terminal and/or non-terminal symbols.
• Each rule is of the form:   A ::= e -> #s
• A is a non-terminal symbol
• e is a parsing expression
• s (optional) is a semantic transformation (data-munging callback)
• Each parsing expression is either: a terminal symbol, a non terminal symbol or the empty string.
• Given the parsing expressions, ++e1++ and ++e2++, a new parsing expression can be constructed using the following operators:
• Sequence: e1 e2
• Ordered choice: e1 / e2
• Zero-or-more: e*
• One-or-more: e+
• Optional: e?
• And-predicate: &e
• Not-predicate: !e

A Conceptual Model of VimPEG

There are three players in the VimPEG game:

1. The VimPEG Parser Generator (Vim plugin)
2. The Language Provider
3. The Client

The VimPEG Parser Generator

This is a Vim plugin you'll need to install to both create and use VimPEG based parsers.

The Language Provider

This is someone who creates a parser for a new or existing language or data-structure. They create the grammar, data-munging callbacks, utility functions and a public interface into their 'parser'.

The Client

This is someone who wants to 'use' a parser to get some real work done. Clients can either be Vim end-users or other VimL coders using a parser as a support layer for even more awesome and complicated higher-level purposes.

There are five pieces to VimPEG

1. The VimPEG library (plugin)
2. A PEG Grammar (provider-side)
3. Callbacks and utility functions [optional] (provider-side)
4. A public interface (provider-side)
5. Client code that calls the provider's public interface. (client-side)

Our Parsing Example

Let's return to our parsing example: recognising (and eventually evaluating) a sum series of integers.

Examples of our expected Input

• 123
• 1 + 2 + 3
• 12 + 34 + 56 + 78

A traditional CFG style PEG for a Series of Integer add & subtract operations:

  Expression  ::= Sum | Integer
  Sum         ::= Integer '+' Expression
  Integer     ::= '\d\+'

In the above PEG for matching a Sum Series of Integers, we have:

• Three non-terminal symbols: 'Integer', 'Sum' and 'Expression'
• Two terminal symbols: \d\+ and  '+'
• One use of Sequence with the three pieces: 'Integer' '+' 'Expression'
• One use of Ordered choice: 'Sum' | 'Integer'

NOTE: The original (and actual) PEG formalism specifies the fundamental expression type as a simple string. VimPEG shuns (at probable cost) this restriction and allows regular expressions as the fundamental expression type. Original PEG grammars use / to indicate choice, but VimPEG uses | instead.

Anyone familiar with CFG grammar specifications will feel right at home with that example PEG grammar above. Unfortunately, it isn't idiomatic PEG. The thing to be parsed here is a list. PEGs have a compact idiomatic way of expressing that structure:

  Expression   ::= Integer (('+' | '-') Integer)*
  Integer      ::= '\d\+'

Here the arguably simpler concept of iteration replaces the CFG use of recursion to describe the desired list syntax to be parsed. It's so much simpler that it seemed a waste not to bundle subtraction in with the deal. Now our parser can evaluate a series of integer add and subtract operations.

The VimPEG API

  peg.e(expression, options)                  "(Expression)
  peg.and(sequence, options)                  "(Sequence)
  peg.or(choices, options)                    "(Ordered Choice)
  peg.maybe_many(expression, options)         "(Zero or More)
  peg.many(expression, options)               "(One or More)
  peg.maybe_one(expression, options)          "(Optional)
  peg.has(expression, options)                "(And Predicate)
  peg.not_has(expression, options)            "(Not Predicate)

Defining the Series of Integer Add and Subtract Operations PEG

  let p = vimpeg#parser({'skip_white': 1})
  call p.e('\d\+', {'id': 'integer'})
  let expression =
        \ p.and(
        \   [ 'integer',
        \     p.maybe_many(
        \       p.and(
        \         [ p.or(
        \           [ p.e('+'),
        \             p.e('-')]),
        \           'integer'])),
        \     p.e('$')],
        \   {'on_match': 'Expression'})

This example demonstrates several aspects of VimPEG's API:

1. Elements that have been 'identfied' (using the 'id' attribute) can be referred to in other expressions. 'Integer' is identified in this case and referenced from 'Expression'.
2. Only root-level elements need to be assigned to a Vim variable. In this case, the 'expression' element is considered to be a root element -- we can directly call on that element now to parse a series of integer add and subtract operations.
3. Intermediate processing (for evaluations, reductions, lookups, whatever) is achieved through callback functions identified by the 'on_match' attribute.  The 'Expression' rule uses such a callback to iterate the list of add or subtract operations to evaluate their final total value. Here is that callback function:

  function! Expression(args)
    " initialise val with the first integer in the series
    let val = remove(a:args, 0)
  
    " remaining element of a:args is a list of [ [<+|->, <int>], ... ] pairs
    let args = a:args[0]
    while len(args) > 0
      let pair = remove(args, 0)
      let val = (pair[0] == '+') ? (val + pair[1]) : (val - pair[1])
    endwhile
    return val
  endfunction

The public API interface

  function! EvaluateExpression(str)
    let res = g:expression.match(a:str)
    if res.is_matched
      return res.value
    else
      return res.errmsg
    endif
  endfunction

The res object holds a lot of information about what was actually parsed (and an errmsg  if parsing failed). The value element will contain the cumulative result of all the 'on_match' callbacks as the input was being parsed.

Using it

  echo EvaluateExpression('123')
  echo EvaluateExpression('1 + 2')
  echo EvaluateExpression('1 + 2 + 3')
  echo EvaluateExpression('4 - 5 + 6')
  echo EvaluateExpression('1 - a')

NOTE: The last example there will return the error message: 'Failed to match Sequence at byte 2'. This might seem unexpected -- we might have been hoping for something more meaningful about not expecting an alphabetic character when looking for an integer digit. It's telling us that (after gracefully falling back out of the optional series of add and subtract operations) it can't match '$' (end of line) at byte 2 because a '-' character is in the way.

Not terribly exciting, granted, but hopefully this serves as a reasonable introduction to the VimPEG Parser Generator. What can you do with it? I look forward to seeing weird and wonderful creations and possibilities in Vim now that real parsing tasks are more accessible.

What's Next?

As beautiful (ok, maybe not, but I've seen more hideous interfaces) as VimPEG's API is, she could do with a touch of lipstick. Instead of calling the API directly, it would be nice to be able to declare the rules using the PEG formalism. That's exactly what Raimondi has done in one of his contributions to VimPEG and that's what we'll be talking about in the next article.

In a future article I will show an example of sugar-coating the VimL language to make function declarations both a little easier on the eyes and fingers as well as adding two long-missing features from VimL -- default values in function parameters and inline function
declarations, a la if <condition> | something | endif .

Now You Know When To Macro

Use macros to reformat multi-line text
that would be too finicky
with :ex's linewise commands.

What Are Macros?

Macros are just recorded keystrokes of what the user would normally do manually when faced with the given editing task. As such, they do not represent much of a conceptual challenge and therefore the barrier to adoption is quite low. Once users gingerly try their first few macros, they're hooked and look for new and interesting screws to bang with their dull new hammer.

Regular expressions, on the other hand, are a mess of complicated hieroglyphics to the layman. Like nothing they've used or done before, they appear opaque and unfathomable. The meaning of various atoms within a regular expression are overloaded (they take on different meanings within different contextual sections of the expression) which only adds to the surprise and frustration of those who make the effort to learn them. Let's not even mention Vim's silly variable magic-ness.

For a great many editing situations, though, regular expressions are the sharpest tool in a Vimmer's toolbox. It slices and dices text cleanly and, when you master them, efficiently. I urge every Vimmer to vault the regex wall; glory awaits you on the other side.

For the right task, however, macros are the right tool - a better tool than regular expressions (alone).. But... what is the right task? In my opinion, macros should be used for multi-line edits where Vim's :ex commands (the :substitute (regular expression driven search & replace command) among them) prove fragile and stubborn.

A quick duck around the net revealed several good explanations and examples of macros being used correctly. It also produced a few cases of macros being used where the better choice would have been one or several regexes.

I won't bore you with yet another contrived example here, I'll simply point to some good ones:

• http://riceball.com/d/node/565
• http://lucapette.com/vim/rails/vim-for-rails-developers-recursive-macros-and-ack/
• http://joaopizani.hopto.org/en/2011/12/vim-macros/

Actually... As I was readying to publish this, I just noticed a macro that I always use. Moving to the first paragraph, I type:

  qavipJ}jq

I then run it on the second paragraph with @a and the subsequent ones with @@. I don't make it crawl over the whole file automatically because the asciidoc multi-line headings, lists and admonition blocks get messed up with this macro. for the length of my articles and the builtin } command to move to the next paragraph, it's not a big deal.

Recursive Macros

One neat trick about macros is that they can be recursive. This is often used to make the macro auto repeat without having to specify a range or count of lines when executing it. While it is fairly trivial to specify a range of lines for a macro to execute over, I'd suspect that if it _were_ that easy then using a macro in the first place was probably the wrong choice. With a macro that utilises a find up front to set the position of the subsequent edits and then calls itself as the last step, you have a simple and neat solution that finds and changes all occurrences for you.

Here is a simple example of a recursive macro:
http://dailyvim.blogspot.com/2009/06/recursive-macros.html

And I've discussed recursive macros before too.


Cases Where Macros Were The Wrong Tool

Of the many poor examples of macros, I'll highlight why I consider macros to be a poor choice with the following two cases (chosen randomly):

• http://www.joeldare.com/wiki/linux:vim_macro

The source data is all on separate, unrelated lines. This is a clear signal that plain :ex (in this case, a regex :substitute) should be employed instead of a macro.

Original source lines:

  First
  Last
  Phone

Desired destination lines:

  <label>First:</label><input type="text" name="First">
  <label>Last:</label><input type="text" name="Last">
  <label>Phone:</label><input type="text" name="Phone">

Simple regex solution:
  :%s/.*/<label>&:<\/label><input type="text" name="&">

• http://www.oreillynet.com/mac/blog/2006/07/more_vim_save_time_with_macros_1.html

Again, the source data is strictly single-line, requiring again, a simple :substitute.

  %s/^\d\+ -- \(.*\)\( (\d\{4})\)$/<li><em>\1<\/em>\2<\/li>/

.Follow up with some normal commands to wrap the list:
  o</ol><esc>
  <c-o>O<ol><esc>


I've tried to show in this article the following things:

• Vim's macros are cool when used in the right place - where the source involves complex, interdependent multi-line text.

• Don't use a macro when the source does not consist of multi-line, interdependent text. Use an :ex command, like :substitute then.

• LEARN regular expressions. They'll save you a LOT of time in the long haul.

The Art of Edits I - Weaponry

When faced with vicious hordes of marauding tweaks, ghastly repeats and soul shaking changes to the landscape that would make a lesser mortal wine and pule under their desks, Vimmers do not wince or cringe or shy away from their keyboards. This is their domain. This is what they've been practising for. The accomplished warrior does not foolhardily rush in to battle without first examining his lot carefully. What is the nature of the beast? What weapon shall I vanquish it with? How many scalps will I collect before my anime starts?

The Armoury

Various weapons adorn the armoury of the Valiant Vimmer:

Edits

A small hand-held weapon, most effective in cramped and dynamic situations. CAUTION: Sole reliance on this weapon will cause fatigue. Remember to sheathe your Insert Edits when you're not using them. These are the normal, insert and visual mode commands that constitute a lot of ordinary editing work. Read the left column of topics in the table of contents of :help quickref for a summary of awesome tips, tricks and techniques this weapon offers.

Abbreviations

Grips and tactics to wield Edits more gracefully.  These are rehearsed and prepared shortenings that automatically expand out to longer forms while we're typing. They're available in both insert and command-line modes. Read :help :abbreviate for the craft and maintenance of this weapon.

Completions

A mystical amulet that bestows upon the wearer the ability to complete an attack automatically. Adept users of this power can call on recent battlefield attacks, those from historic battles, from battlefield manuals or even from an omniscient guiding hand. Read :help ins-completion for access to this gem of valour.

Regexes

The most devastating hand-held weapon in the Vimmer's arsenal. Due to its ungainly size and the need to retreat from battle while preparing an offensive with this weapon, it is not recommended for close quarter or highly dynamic combat. Often best suited for ranged attacks. The adept Vimmer is an effective user of Vim's regular expressions. To sharpen your regex skills, read :help pattern.txt and you can practise with VimRegexTutor.

Ex

One of a warrior's greatest strengths is the ability to command his troops amidst the frenzy of battle. Ex is the powerful tool Vimmers use to coordinate and execute larger strategies on the battlefield. Raining fire down upon your enemies from afar, changing the very landscape beneath their feet, and even travelling in time are but a few of the awesome powers at the command of a Vimmer versed in Ex. These are all of the : (colon) commands in Vim. These commands are inherently linewise and can therefore be ungainly on multiline monsters (macros are the recommended weapon when facing multi-lined monstrosities). Read :help ex-cmd-index for an overview of battlefield orders you can issue.

Maps

The map is a ranged weapon providing a sortie of commands with the flick of a wrist. Skilled Vimmers spend hours in training creating map weapons for special purpose attacks. The builtin Edits occupy a vast majority of the available body (keyboard) space.  Careless novices unwittingly replace valuable original Edits with inferior maps, only to be defeated or fatigued in extreme battle.  The master Vimmer chooses his maps carefully, regularly preferring to place them on the purpose-built shield (:help mapleader) so as not to interfere with his perfected mastery of the native Edits. Read :help map.txt to start crafting your own maps of devastation.

Macros

Macros are pre-rehearsed battle maneuvers that can be performed instantly with precise execution. A powerful weapon but dangerous when employed on the wrong terrain. The macro will blindly repeat the instructions it was established with. Regardless of whether this weapon is fired when facing the wrong enemy or even friendly troops, it will charge headlong in to battle until its target is demolished or it has been overcome (an error occurs).  Here are several good explanations of macros and their uses.

VimL

The arcane language of the truly wizened Vimmer, capable of wreaking havoc on even the most worthy opponent. Once a spell is cast its effects are almost instantaneous, but the incantation process is long and complicated. This discipline is better practised at quieter times of relative peace whereupon its power may be distilled into volatile potions, sacred scrolls, charged wands and ready tomes that can be more freely and immediately used on the battlefield. Such artefacts in the land of Vim are called plugins.  To begin on your path as a Vim Mage, read Damian Conway's excellent introduction to Vimscript. The accomplished Vimlglot might prefer a deeper analysis. The holy arcanum is kept in :help usr_41.txt.

Veteran Vimmers and Alacritous Acolytes alike are well advised to review their personal arsenal frequently, polishing tarnished weapons and sharpening dull and forgotten ones. Develop a regimen of daily practise in each of these tools and skills so that you may face your next editing evil with brave heart, right mind and quick hand.

Being virtuous in preparation and valiant in battle,
the nature of a Vimmer is irrepressible!

Advanced Macros in Vim, Commentary

This is a bloated comment on: http://blog.sanctum.geek.nz/advanced-vim-macros/

Sharks in Shells

Firstly, there are at least two plugins that can tabulate text in vim: Align and Tabular. Using plugins written in pure vimscript is usually better than shelling out to system tools, because it's more OS independent. On that same note, Vim has a builtin :sort command, so no need to shell out for that either.

You warn against reaching for the hammer when holding screws; that it's sometimes better to use a splash of :ex and a sprinkle of VimL. Indeed, that is true in this example (starting with the cursor on the first line of the original data table):

  :Tabularize / \+/
  :sort
  :%s/\d\{4}/\=strftime('%Y')-submatch(0)/

Mentioning Macro Mechanics

Yanking a macro line from your buffer into a register with "ayy captures the trailing ^J which, if you have something bound to <Enter>, will produce unwanted side-effects. Use ^"ay$ instead (or the VimLocalMacros plugin).


When assigning macros in a let expression, the double-quote form does allow for Vim map-style markup:

  let @a = 'must have ^M literal chars'
  let @a = "can have \<cr> escaped markup"

There's no inherent benefit from moving this sort of macro into a function - it's so uniquely dependent on your current text file (and year!) that it simply doesn't make sense in this case. However, this trick might be useful in more generic situations. Using a function to preserve macro contents allows us to reuse the register that macro was occupying. Personally, I use macros on an ad hoc basis, crafting them quickly for the need at hand and discarding them when done. I might overwrite register @a several times over within a single edit session with various macros on an as needed basis. However, I was asked one day how to persist particular macros for a given file. The embedded-in-a-function approach shown in the article is one solution. I also wrote VimLocalMacros for this purpose.

Macros stop executing if an error occurs. You can use this to your benefit. Design your macros so they deliberately fail - easy way to not have to count and just provide a large-enough number of runs: 999@a

Recursive Macros have a use and are not something to be dismissed as the wrong tool instead of reaching for a scripting language.