Monday, April 23, 2012

Undo & Continue Confirmed Substitution

Janni on #vim asks:

When using substitution (:s) with the [c] flag one can use several keys to
confirm/skip concrete substitutions. I'd really like to get the additional
option to undo the last substitution.  (without exiting the substitution
process as a whole)


Here's how I would handle that situation:

For demonstration purposes, let's replace "e" with "z".

We have to start on the first line, so type   gg

The following command operates on all lines from the cursor to the end of the
file which contain an "e":

:.,$g/e/s//z/gc

The /c on the :substitute command there prompts us at every replace position. See :help :s_flags for an explanation of the interactive options.

If you want to undo the previous one, press   ctrl-c   to cancel the :global
command and then press   u   to undo and   j   to move down a line (skipping
over the line you just undid so as not to be asked about it again). Then re-run
the   :.,$g/e/s//z/gc   command. The easiest way to do that is to press
:<up><enter>

It'd be nice to have an extra   u   option in the interactive confirmed
substitution command to undo the last replacement. Until then, enjoy.

Update:

The ever  vigilant Raimondi caught me with my pants down again. It turns out, we don't need the extra :g// at all here. A simple:

:.,$s/e/z/gc

is sufficient. It has exactly the same runtime mechanics as the method described above — that is, use ctrl-c to cancel the operation midway, u to undo changes and then :<up><enter> to recommence the substitute from the current line onwards. Thanks for keeping me honest, Raimondi. <3

Your gem, my rock:

Sir Raimondi followed up with another eloquent query:

y u no @: bairui?!

Ah... good question. Ostensibly, I didn't show @: because I rarely use it myself. For my needs, :<up><cr> is in the muscle memory... Besides, @: is not necessarily easier for me to reach. Also, it gives me a chance to eye-ball it before accidentally re-running the wrong command. My supply of brown underwear is limited so it behooves me to be a little cautious around my :ex.

But yes, @: and then subsequent @@ is faster if you know you have to re-run an :ex command many times, as is the case here... but if you're having to do that a lot manually... I smell fishies... :-) Of course, I just polluted your wading pond with a load of carp by suggesting you do exactly that above — what can I say? When you know the rules, you can break the rules?   :-p   Love as always, R.

Sunday, April 22, 2012

Vim WAZ 'ere

Sir Raimondi and I have started a project of converting the venerable vi/ex tutorial by Walter Alan Zintz to Vim.

The first three chapters have been converted and styled (with asciidoc).

Feedback welcome.

Monday, April 16, 2012

Lengthly

It's time again for another comparison of approaches in Vim. This time, we're looking at the question:

How do you sort a file by line length?

Our Dataset

  one
  two
  three
  four
  five
  six
  seven
  eight
  nine
  ten
  a line with twenty nine chars
  one with 11

The Clumsy Ex

Optionally select just the range to work on, either visually or explicitly.  Here, I will assume the whole file is to be line-lengthly sorted.

  :%s/^/\=len(getline('.')) . " "
  :sort n
  :%s/^\d\+ //

Yielding

  one
  two
  six
  ten
  four
  five
  nine
  three
  seven
  eight
  one with 11
  a line with twenty nine chars

NOTE: This is clumsy for two reasons:
  1. It requires messing with the buffer text. Three separate operations are needed to complete this single logical task.
  2. The sort considers only the numeric value and not the subsequent textual values, resulting in lines indeed sorted by line length, but left in an apparently random disarray otherwise.
Can we have it all? Can we sort by line length, and then subsequently alphabetically, all in the one operation?

Let VimL Light Your Way

First, we'll need a comparator. The sort() function does so on the textual representation of the data passed to it by default. This can be overridden by passing the name of a function as the second argument. Such a named function looks like this:

  function! Lengthly(i1, i2)
    let li1 = len(a:i1)
    let li2 = len(a:i2)
    return li1 == li2 ? (a:i1 == a:i2 ? 0 : (a:i1 > a:i2 ? 1 : -1)) : li1 > li2 ? 1 : -1
  endfunction

Or the more verbose equivalent (with less eye-bleeding ?: statements):

  function! Lengthly2(i1, i2)
    let i1 = a:i1
    let i2 = a:i2
    let len_i1 = len(i1)
    let len_i2 = len(i2)
    if len_i1 == len_i2
      if i1 == i2
        return 0
      else
        if i1 > i2
          return 1
        else
          return -1
        endif
      endif
    else
      if len_i1 > len_i2
        return 1
      else
        return -1
      endif
    endif
  endfunction

*shudder* - while the ?: mess above is arguably hard to read, I find the more verbose form even more distracting.

NOTE: I have a penchant for naming my sort comparators as adverbs like that - I like how they read in the subsequent   sort([], 'Lengthly')   call.

So, armed with our new comparator, let's sort some lines.

Across the whole file (as in the earlier example):

  :call setline(1, sort(getline(1, '$'), 'Lengthly'))

Over a visually selected range (using explicit range end markers):

  :<c-u>call setline("'<", sort(getline("'<", "'>"), 'Lengthly'))

NOTE: The   <c-u>   is there to clear the   '<,'>   visual range markers Vim helpfully (though, unnecessarily in this case) inserts for us when pressing   :   while a visual selection is in effect.

Over a visually selected range (using the selection register):

  :<c-u>call setline("'<", sort(split(@*, "\n"), 'Lengthly'))

NOTE: You will need   :set clipboard=autoselect   to have the @* register auto-populated with the current visual selection.

The Result

  one
  six
  ten
  two
  five
  four
  nine
  eight
  seven
  three
  one with 11
  a line with twenty nine chars

Beautiful! No messy manipulations of the lines (inserting and deleting line lengths on each line) before and after sorting AND not only are the lines sorted by line length but they're in alphabetical order within their length groups too. VimL, I <3 you.

Technically, I could have cobbled together a long-winded vanilla VimL function, but 30 minutes into this post, I suddenly got very bored.

Gotchas for the weary

I had pasted (an admittedly utterly goatse) version of the VimL solution up to #vim before I thought about blogging this. It was an overly complicated spaghetti mess of maps and sort and other assorted sins. Despite its debatable beauty, it was just plain WRONG. It failed to respect sort()'s preference to sort textually, resulting in files that had 1 and then 11 and then 2 line lengths in a decidedly unappealing jaggy edge.

Lesson Learned: Test on larger data sets than the English numbers one to ten. :-p

As always, it's not the destination with these sort of posts, but rather the journey. Hope you enjoyed the ride. Now... you can walk home.

Before You Go

I generate my test data with a little map I keep handy for just this purpose:

   :nnoremap <silent> <leader>T :<c-u>call append('.', map(range(v:count1), 'NumToNumber((1+v:val))'))<CR>

With the keychord   25<leader>T   I generate 25 lines of test data beneath my cursor. Sure, as mentioned just above, testing on textual representations of English numbers can sometimes be limiting, but this is the first time I've noticed the bite. The magical function   NumToNumber()   comes from my firstly plugin which contains such gems as:
  • NumToNumber(num)        " 1 -> one
  • NumToOrd(num)           " 1 -> 1st
  • NumToOrdinal(num)       " 1 -> first
  • NumberToNum(engnum)     " one -> 1
  • NumberToOrd(engnum)     " one -> 1st
  • NumberToOrdinal(engnum) " one -> first
  • OrdToNum(ord)           " 1st -> 1
  • OrdToNumber(ord)        " 1st -> one
  • OrdToOrdinal(ord)       " 1st -> first
  • OrdinalToNum(engnum)    " first -> 1
  • OrdinalToNumber(engnum) " first -> one
  • OrdinalToOrd(engnum)    " first -> 1st
Enjoy. :-)

Sunday, April 15, 2012

Do it with Vim, Nexus

Today I needed to rename a bunch of mp3 files in the format split-track01.mp3 to their real names as described in a .cue file. All the heavy lifting of splitting the actual music files out of the .ape were handled by cuebreakpoints and shntool. Even the tag data was marshalled over nicely with cuetag. If I had known about the -t flag on shntool's split subcommand, my day would have been over and done with. I didn't, so I was faced with a bunch of uselessly named .mp3 files with the goodies buried in a .cue file.

An exercise for the user! Joy! Well, we know we ain't gonna do that by hand.  So... I could fire up perl or ruby or awk or just sed... Bah. Vim it is then!

I gingerly cradled my freshly microwaved mug of cocoa while pondering how to eat this elephant. Deciding it was too hot yet to enjoy, I jumped into the terminal and typed:

  vim file.cue

Then with a flick of the Vim wrist, I mashed:
 
  :v/^\s\+TITLE/d
  :%s/^\s*TITLE/\=printf("mv split-track%02d.mp3", s1.next())
  :%s/"$/.mp3"/
  :set fileformat=unix
  :saveas rename.sh
  :quit

Ah, almost done. Now, out in the terminal again, I just:

  chmod +x ./rename.sh
  ./rename.sh

And I still had to wait for my cocoa to cool.

NOTE: The fileformat=unix nonsense was because the original had dos line endings.

The pious reader will have noticed a fairly bold cheat in my Vim there. That s1 object looks suspicious. It's not as heinous as you might imagine. He belongs to my Nexus plugin which provides series objects for just these sort of occasions in Vim.

The s1 object has a method called next() which returns the next number in the series. As his name implies, the first such number is 1. There is a corresponding s0 object whose first call to next() yields, of course, the number 0. The gentle user is free to create their own series objects too:

  :let even_seq = Series(0, 2)

The command:

  :echo even_seq.next()

will print 2 on the first call, then 4, then 6, 8, 10, etc, on subsequent calls.

The Series constructor has the following signatures:
  • Series() -> starting at 0 with an increment of 1 -> 1, 2, 3, 4, ...
  • Series(7) -> starting at 0 with an increment of 7 -> 7, 14, 21, 28, ...
  • Series(7, 2) -> starting at 7 with an increment of 2 -> 9, 11, 13, 15, ...
  • Series(2, 7) -> starting at 2 with an increment of 7 -> 9, 16, 23, 30, ...
Nexus's Series Objects provide the following methods:
  • reset() to set the object back to its original values when created
  • next() to increment and return the next number in the series
  • inc() as an alias for next()
  • value() to return the current value without incrementing internal state
  • val() as an alias for value()
  • set() to explicitly alter the internal value (but not increment amount)
I use s1 a lot in my daily vimming. It could be useful to you too.

Saturday, April 14, 2012

WAZ - Vim Editor Fundamentals

This is not my work. I stole it. Well, borrowed, really.
Walter Alan Zintz wrote the original for the venerable vi editor circa '96. I first read it in '99. It was awesome then; it still is now. Today, I'm dusting it off and rubbing a bit of vim polish in to highlight this timeless tutorial.  Enjoy.

(Apologies, Walter, if you find this distasteful... I tried to contact you at walter@ccnet.com but I got a delivery failure notice :-/ )

I aim to change only the editor commands to vimify the original vi. I may have to change some of the description in the accompanying prose to accommodate. I hope that's okay with everyone.

Throughout  this tutorial, Walter refers to visual mode and line mode, which may have confusing meanings to existing Vimmers. By visual mode, Walter is not referring to Vim's visual selection mode, but rather to Vim's full-window (aka normal) mode where you can see a screenful of your file all at once. This is in contrast to vi's notion of line mode wherein you see and operate on a line (or range of lines) through explicit line addresses ONLY. Vim still possesses this ancient mode but refers to it as Ex mode. To see it, type Q in normal mode, or the marginally friendlier gQ.  After both minutes of intrigue, excitement and wonder pass, type vi[sual] to return to normal Vim mode (which, remember, Walter refers to as visual mode, and now you know why.)

And now, over to Walter:

Why Vim?
A HEARTWARMING EDIT

Pity poor Hal, a corporate maintenance programmer. A large module of badly-broken, poorly-patched legacy code -- the spaghetti variety -- finally broke down completely yesterday, leaving one corporate division running at half speed. By dint of some inspired fixes during an all-nighter, Hal has the module up and running again this morning... but just as he's ready to go out for food that isn't from a vending machine, in walks the corporation's VP of IS, with a big surprise.

"Nice work on that crash fix, Hal; but right now I need some formatted technical data about it, in a hurry. The Board of Directors' Information Systems Committee has called a rush meeting this morning to convince themselves they're on top of the problem. I'll be in the hotseat, and I need technical data I can put up on the video projector to keep them occupied.

"They'll want me to discuss the logfile of errors that led up to the crash...  yes, I know that's in /oltp/err/m7, but appending puts the latest report lines at the bottom of the file. Those suits aren't interested in what they think is ancient history, and they wouldn't be caught reading anything but a commuter train timetable from the bottom up, so you'll have to make a copy with the order of the lines reversed: what was the last line becomes the first line, what was the second to the last line is now line number two, and so on.

"And let's take a look at that logfile.

  374a12  44872  130295/074457  nonabort
  5982d34  971  130295/221938  nonabort
  853f7  2184  140295/102309  abort
  ...

Hmmm. Explaining the second column to them would be advertising the fact that we knew this failure was just waiting for a chance to happen. So while you're at it, go through and erase all but the first and last digits of each number in column two.

"Oh, and when they get tired of that they'll want to scrutinize the Lint report. Last month I told them that our Lint substitute was the greatest thing since Marilyn Monroe, so now they'll want me to tell them why the messages it still generates on this module aren't real hazards. Just run Lint over the revamped module; then combine the Lint output with a copy of the source file by taking each message line like:

  Line 257: obsolete operator +=

and putting the significant part at the end of the source line it refers to.  And put a separator, like XXX, between the source line and the message so I can page through quickly. Nothing like a hefty dose of source code they can't begin to fathom to make the meeting break up early.

"And get right on this. The meeting starts in 35 minutes."

Our VP walks away inwardly smiling, thinking he's getting out of detailed explanations and putting all the blame on an underling, just by demanding more editing than anyone could do in the time available. "I'll tell the Information Systems Committee that I made it perfectly clear to the programmer that we needed this at 9:30, but when I asked him for it a minute ago he said it wasn't finished and he wasn't sure when it would be. Then I'll remark that those programmers just can't understand that keeping management informed is every bit as important as writing code!"

But Hal has a secret weapon against this squeeze play: an expert knowledge of the Vim editor.

Reversing the order of the lines in a file is a piece of cake with this editor.  The eight keystrokes in:

  :g/^/m0

Bazz's Note: Commands in Vim starting with : need to be finished by pressing RETURN.

will do it. Taking the digits out of the middle of the second column throughout the file also requires just one command line:

  :%s/\v +\zs(\d)\d+(\d)/\1\2

And integrating the Lint messages into a copy of the source code? Even that can be automated with the Vim editor. The editor command:

  :%s/\vLine (\d+): (.*)/\1s;$; XXX \2

will turn that file of Lint messages into an editor script, and running that script on a copy of the source file will mark it up as requested.

Rather than being portrayed as a bungler, Hal can have it all ready in a couple of minutes, just by typing a few lines. He'll even have time to guard against vice-presidential prevarication, by disappearing into the coffee shop across the street and reappearing just as the meeting is getting started, to tell the VP (and everyone else in earshot), "Those files you wanted are in slash-temp-slash-hal".

THE PLAN OF THIS ONGOING TUTORIAL

I'm writing here for editor users who have some fluency in Vim at the surface level. That is, you know how to do the ordinary things that are belabored in all the "Introducing Vim" books on the market, but rarely venture beyond that level.

This tutorial series will explore a lot of other capabilities that hardly anyone knows are in Vim. That includes quite a few tricks that may be built on editor functions we all use every day, but which nonetheless are not obvious -- for instance, telling the global command to mark every line it encounters.  I'll also be clarifying the real nature of the many misunderstood aspects of this editor.

To do all this, I'll be explaining things in more depth than you might think warranted at first. I'll also throw in exercises wherever they seem helpful.  And to save you readers from gross information overload, I'll write this tutorial in a large number of fairly small modules, to be put up on our website at a calm, reasonable pace.

Bazz's Note:   :s/our/my/




The Editor's Basic Concepts

To get a real grasp on this editor's power, you need to know the basic ideas embodied in it, and a few fundamental building blocks that are used throughout its many functions.

One cause of editor misuse is that most users, even experienced ones, don't really know what the editor is good at and what it's not capable of. Here's a quick rundown on its capabilities.

First, it's strictly a general-purpose editor. It doesn't format the text; it doesn't have the handholding of a word processor; it doesn't have built-in special facilities for editing binaries, graphics, tables, outlines, or any programming language except Lisp.

It's two editors in one. Visual mode is a better full-screen editor than most, and it runs faster than those rivals that have a larger bag of screen-editing commands. Line editing mode dwarfs the "global search and replace" facilities found in word processors and simple screen editors; its only rivals are non-visual editors like Sed where you must know in advance exactly what you want to do. But in the Vim editor, the two sides are very closely linked, giving the editor a combination punch that no other editor I've tried can rival.

Finally, this editor is at its best when used by people who have taken the trouble to learn it thoroughly. It's too capable to be learned well in an hour or two, and too idiosyncratic to be mastered in a week, and yet the power really is in it, for the few who care to delve into it. A large part of that power requires custom-programming the editor: that's not easy or straightforward, but what can be done by the skillful user goes beyond the direct programmability of any editor except (possibly) Emacs.

Search Patterns

In quite a few functions of this editor, you can use string-pattern searching to say where something is to be done or how far some effect is to extend. These search patterns are a good example of an editor function that is very much in the Unix style, but not exactly the same in detail as search patterns in any other Unix utility.

Search patterns function in both line editing and visual editing modes, and they work the same way in both, with just a few exceptions. But how you tell the editor you're typing in a search pattern will vary with the circumstances.

SEARCHING FROM WHERE YOU ARE NOW

The more common use for search patterns is to go to some new place in the file, or make some editing change that will extend from your present position to the place the pattern search finds. (In line editing mode it's also possible to have an action take place from one pattern's location to where another pattern is found, but both searches still start from your present location.)

If you want to search forward in the file from your present location (toward the end of the file), precede the search pattern with a slash ( / ) character, and type another to end the pattern. So if you want to move forward to the next instance of the string "j++" in your file, typing:

  /j++/

will do it. And so will:

  /j++

When there is nothing between the pattern and the RETURN key, the RETURN itself will indicate the end of the search pattern, so the second slash is not necessary.

To search backward (toward the start of the file), begin and end with a question mark instead of a slash. The same rules of abbreviation apply to backward searches, so:

  ?j++?

and:

  ?j++

both head backward in the file to the same pattern.

Either way, you've expressed both your request for a pattern search and the direction the search is to take in just one keystroke. But don't assume that if you search backward, any matching pattern the editor finds will be above your present position in the file, and vice versa if you search forward. The editor looks there first, certainly, but if it gets to the top or bottom line of the file and hasn't found a match yet, it wraps around to the other end of the file and continues the search in the same direction. That is, if you used a question mark to order a backward search and the editor searches all the way through the top line of the file without finding a match, it will go on to search the bottom line next, then the second-to-the-bottom line, and so on until (if necessary) it gets back to the point where the search started. Or if you were searching forward and the editor found no match up through the very last line of the file, it would next search the first line, then the second line, etcetera.

If you don't want searches to go past either end of the file, you'll need to type in a line mode command:

  :set nowrapscan

This will disable the wraparound searching during the present session in the editor. If you want to restore the wraparound searching mechanism, typing:

  :set wrapscan

will do it, and you can turn this on and off as often as you like.

THE FIND-THEM-ALL SEARCH

Up to now, I've been considering searches that find just one instance of the pattern; the one closest to your current location in the file, in the direction you chose for the search. But there is another style of search, used primarily by certain line editing mode commands, such as global and substitute. This search finds every line in the file (or in a selected part of the file) that contains the pattern and operates on them all.

Don't get confused when using the global and substitute commands. You'll often use both styles of search pattern in one command line. But the find-one-instance pattern or patterns will go before the command name or abbreviation, while the find-them-all pattern will come just behind it. For example, in the command:

  :?Chapter 10?,/The End/substitute/cat/dog/g

the first two patterns refer to the preceding line closest to the current line that contains the string "Chapter 10" and the following line closest to the current line containing the string "The End". Note that each address finds only one line. Combined with the intervening comma, they indicate that the substitute command is to operate on those two lines and all the lines in between them. But the patterns immediately after the substitute command itself tell the command to find every instance of the string "cat" within that range of lines and replace it with the string "dog".

Aside from the difference in meaning, the two styles also have different standards for the delimiters that mark pattern beginnings and (sometimes) endings. With a find-them-all pattern, there's no need to indicate whether to search forward or backward. Thus, you aren't limited to slash and question mark as your pattern delimiters. Almost any punctuation mark will do, because the editor takes note of the first punctuation mark to appear after the command name, and regards it as the delimiter in that instance. So:

  :?Chapter 10?,/The End/substitute;cat;dog;g

  :?Chapter 10?,/The End/substitute+cat+dog+g

  :?Chapter 10?,/The End/substitute{cat{dog{g

are all equivalent to the substitution command above. (It is a good idea to avoid using punctuation characters that might have a meaning in the command, such as an exclamation point, which often appears as a switch at the end of a command name.)

The benefit of this liberty comes when the slash mark will appear as itself in the search pattern. For example, suppose our substitution command above was to find each pair of consecutive slash marks in the text, and separate them with a hyphen -- that is, change // to /-/ . Obviously:

  :?Chapter 10?,/The End/substitute/////-//g

won't work; the command will only regard the first three slashes as delimiters, and everything after that as extraneous characters at the end of the command.  This can be solved by backslashing:

  :?Chapter 10?,/The End/substitute/\/\//\/-\//g

but this is even harder to type correctly than the first attempt was. But with another punctuation mark as the separator:

  :?Chapter 10?,/The End/substitute;//;/-/;g

the typing is easy and the final command is readable.

SIMPLE SEARCH PATTERNS

The simplest search pattern is just a string of characters you want the editor to find, exactly as you've typed them in. For instance: "the cat". But, already there are several caveats:

This search finds a string of characters, which may or may not be words by themselves. That is, it may find its target in the middle of the phrase "we fed the cat boiled chicken", or in the middle of "we sailed a lithe catamaran down the coast". It's all a matter of which it encounters first.

Whether the search calls "The Cat" a match or not depends on how you've set an editor variable named ignorecase . If you've left that variable in its default setting, the capitalized version will not match. If you want a capital letter to match its lower-case equivalent, and vice versa, type in the line mode command:

  :set ignorecase

To resume letting caps match only caps and vice versa, type:

  :set noignorecase

The search absolutely will not find a match where "the" occurs at the end of one line and "cat" is at the start of the next line:

  and with Michael's careful help, we prodded the
  cat back into its cage.  Next afternoon several

It makes no difference whether there is or isn't a space character between one of the words and the linebreak. Finding a pattern that may break across a line ending is a practically impossible task with this line-oriented editor.

Where the search starts depends on which editor mode you're using. A search in visual mode starts with the character next to the cursor. In line mode, the search starts with the line adjacent to the current line.

METACHARACTERS

Then there are search metacharacters or "wild cards": characters that represent something other than themselves in the search. As an example, the metacharacters . and * in:

  /Then .ed paid me $50*!/

could cause the pattern to match any of:

  Then Ted paid me $5!

  Then Red paid me $5000!

  Then Ned paid me $50!

or a myriad of other strings. Metacharacters are what give search patterns their real power, but they need to be well understood.

To understand these, you must know the varied uses of the backslash ( \ ) metacharacter in turning the "wild card" value of metacharacters on and off.

In many cases, the meta value of the metacharacter is on whenever the character appears in a search pattern unless it is preceded by a backslash; when the backslash is ahead of it the meta value is turned off and the character simply represents itself. As an example, the backslash is a metacharacter by itself, even if it precedes a character that never has a meta value. The only way to put an actual backslash in your search pattern is to precede it with another backslash to remove its meta value. That is, to search for the pattern "a\y", type:

  /a\\y/

as your search pattern. If you type:

  /a\y/

the backslash will be interpreted as a metacharacter without any effect (since the letter y is never a metacharacter) and your search pattern will find the string "ay".

Less-often-used metacharacters are used in exactly the opposite way. This sort of character represents only itself when it appears by itself. You must use a preceding backslash to turn the meta value on. For example, in:

  /\<cat/

the left angle bracket ( < ) is a metacharacter; in:

  /<cat/

it only represents itself. These special metacharacters are pointed out in the list below.

Finally there is a third class, the most difficult to keep track of. Usually these metacharacters have their meta values on in search patterns, and must be backslashed to make them represent just themselves: like our first example, the backslash character itself. But if you've changed the default value of an editor variable named magic to turn it off, they work oppositely -- you then must backslash them to turn their meta value on: like our second example, the left angle bracket. (Not that you are are likely to have any reason to turn magic off.) These oddities are also noted in the list below.

And don't forget the punctuation character that starts and ends your search pattern, whether it is slash or question mark or something else. Whatever it is, if it is also to appear as a character in the pattern you are searching for, you'll have to backslash it there to prevent the editor thinking it is the end of the pattern.

TABLE OF SEARCH PATTERN METACHARACTERS

.

A period in a search pattern matches any single character, whether a letter of the alphabet (upper or lower case), a digit, a punctuation mark, in fact, any ASCII character except the newline. So to find "default value" when it might be spelled "default-value" or "default/value" or "default_value", etcetera, use /default.value/ as your search pattern. When the editor variable magic is turned off, you must backslash the period to give it its meta value.

*

An asterisk, plus the character that precedes it, match any length string (even zero length) of the character that precedes the asterisk. So the search string /ab*c/ would match "ac" or "abc" or "abbc" or "abbbc", and so on. (To find a string with at least one "b" in it, use /abb*c/ as your search string.) When the asterisk follows another metacharacter, the two match any length string of characters that the metacharacter matches. That means that /a.*b/ will find "a" followed by "b" with anything (or nothing) between them. When the editor variable magic is turned off, you must backslash the asterisk to give it its meta value.

^

A circumflex as the first character in a search pattern means that a match will be found only if the matching string occurs at the start of a line of text. It doesn't represent any character at the start of the line, of course, and a circumflex anywhere in a search pattern except as the first character will have no meta value. So /^cat/ will find "cat", but only at the start of a line, while /cat^/ will find "cat^" anywhere in a line.

$

A dollar sign as the last character in a search pattern means the match must occur at the end of a line of text. Otherwise it's the same as circumflex, above.

\<

A backslashed left-angle bracket means the match can only occur at the start of a simple word (In this editor, a "simple" word is either a string of one or more alphanumeric character(s) or a string of one or more non-alphanumeric, non-whitespace character(s), so "shouldn't" contains three simple words.) Thus /\<cat/ will find the last three characters in "the cat" or in " tom-cat", but not in "tomcat". To remove the meta value from the left angle bracket, remove the preceding backslash: /<cat/ will find "<cat" regardless of what precedes it.

\>

A backslashed right angle bracket means the match can occur only at the end of a simple word. Otherwise the same as the left angle bracket, above.

~

The tilde represents the last string you put into a line by means of a line mode substitute command, regardless of whether you were in line mode then or ran it from visual mode by preceding it with a colon ( : ). For instance, if your last line mode substitution command was s/dog/cat/ then a /the ~/ search pattern will find "the cat". But the replacement string of a substitute command can use metacharacters of its own, and if your last use involved any of those metacharacters then a tilde in your search pattern will give you either an error message or a match that is not what you expected. When the editor variable magic is turned off, you must backslash the tilde to give it its meta value.

Bazz's Note: Vim also has a verymagic mode which can be enabled within patterns using the \v metacharacter. This verymagic mode was used in Hal's story earlier. See :help magic for a thorough explanation of Vim's magic, nomagic and verymagic modes.

CHARACTER CLASSES

There is one metastring form (a "multicharacter metacharacter") used in search patterns. When several characters are enclosed within a set of brackets ( [] ), the group matches any one of the characters inside the brackets. That is, /part [123]/ will match "part 1", "part 2" or "part 3", whichever the search comes to first. One frequent use for this feature is in finding a string that may or may not be capitalized, when the editor variable ignorecase is turned off (as it is by default). Typing /[Cc]at/ will find either "Cat" or "cat", and /[Cc][Aa][Tt]/ will find those or "CAT". (In case there was a slip of the shift key when "CAT" was typed in, the last pattern will even find "CaT", "CAt", etcetera.)

There's more power (and some complication) in another feature of this metastring: there can be metacharacters inside it. Inside the brackets, a circumflex as the first character reverses the meaning. Now the metastring matches any one character that is NOT within the brackets. A /^[^ ]/ search pattern finds a line that does not begin with a space character. (You're so right if you think that the different meta values of the circumflex inside and outside the character class brackets is not one of the editor's best points.) A circumflex that is not the first character inside the brackets represents just an actual circumflex.

A hyphen can be a metacharacter within the brackets, too. When it's between two characters, and the first of the two other characters has a lower ASCII value than the second, it's as if you'd typed in all of the characters in the ASCII collating sequence from the first to the second one, inclusive. So /[0-9]%/ will find any numeral followed by the percent sign ( % ), just as /[0123456789]%/ would. A /[a-z]/ search pattern will match any lower-case letter, and /[a-zA-Z]/ matches any letter, capital or lower case. These two internal metacharacters can be combined: /[^A-Z]/ will find any character except a capital letter. A hyphen that is either the first or the last character inside the brackets has no meta value. When a character-hyphen-character string has a first character with a higher ASCII value than the last character, Vim complains with the error message:

  E16: Invalid range.

Backslashing character classes is complex. Within the brackets you must backslash a right bracket that's part of the class; otherwise the editor will mistake it for the bracket that closes the class. Of course you must backslash a backslash that you want to be part of the class, and you can backslash a circumflex at the start or a hyphen between two characters if you want them in the class literally and don't want to move them elsewhere in the construct.  Elsewhere in a search pattern you will have to backslash a left bracket that you want to appear as itself, or else the editor will take it as your attempt to begin a character class. Finally, if magic is turned off, you'll have to backslash a left bracket when you do want it to begin a character class.

Coming Up Next

In the second part of this tutorial, I'll be following up on all this information about search patterns, by showing the right ways to combine them with other elements to generate command addresses.

The Ende (of Walter's Vi Tutorial, Part I)

Now, I don't know if I should continue converting the remaining eight parts of Walter's original vi tutorial to Vim. For now, you can enjoy it in its original glory. The subsequent chapters get quite exciting wherein Walter shows you how to get your vi ninja on.

Sunday, April 8, 2012

macronomicron

The only thing better than macros in Vim are recursive macros.*

(* Actual betterness may not apply.)

Cryptic Overview for the Gifted

1. first - use recursive macros
2. stop the ride - have an error condition
3. the quaker - qaqqa
4. secret sauce - j0@aq
5. blow the raspberry: @

Slightly Deeper Explanation for the Rest of Us

Macros are cool. They are referred to as complex repeats in the Vim docs. See  :help q   for the mechanics of recording and then read on for playing back macros. For the lazy, here is a brief recap:

q{0-9a-zA-Z"}   begins recording into the specified register. All movements, insertions, deletions, :ex commands, etc, are recorded until you press   q   again. So, the command   qagUwWq   creates a macro that upper-cases the current word (from the cursor forward) and then moves to the next WORD (skipping whitespace and punctuation). To run this macro on the next word, we use the command   @a   and after that, to run it on another word we only need to use   @@   (which repeats the last macro command, conveniently.)

If you've already got a partial command in a lettered register (a-z), you can append more command to it by using the   q   command with the upper-case form of the lettered register (A-Z)

There are quite a few other nifty tricks you can pull with macros, so I implore you to head on over to   :help :@   and guzzle that goodness up. But here, I'm going to continue talking about recursive macros - macros that call themselves. Recursive macros are one way to repeat a series of commands across multiple chunks of text all automatically so you don't have to hit   @a   or even   @@   over and over and over... your @ key thanks you now.

Stop the ride

If you don't provide an action within the body of your macro that causes an error, the macro will run from the point of execution until the end of the file. This may be what you want; it may not. To have it stop somewhere specific, you'll have to ensure it errors at that point. Using the    FfTt   commands are a common way to force a failed search. If you use a pattern search with the   /   command, you may need to use anchors to cause failures.

The quaker

Because a recursive macro executes itself with the command   @a   (assuming the macro is being recorded in register   a), then at the point in time that the macro is being created, the contents of register   a   *before* this recording began will be inserted into the macro body when it sees   @a. Unless you know what you're doing, you probably want this to be empty. To this end, we use   qaq   as a fast way to empty register   a. By recording nothing into   a, we effectively empty it. The subsequent   qa   in the command string   qaqqa   begins the real recording.

Secret Sauce

To allow the recursive invocation of your macro to be actually useful,  you have to arrange for it to start in the right place. This could be on the next line, or at the next word, or next paragraph, or block, or... etc. The command   j0    moves down a line and to the first column of the screen, ready for the subsequent recursive invocation with the command   @a   and switching off recording with   q   hence   j0@aq

Blow the Raspberry

You're done recording. Now you need to run the sucker. We do that with   @a   (presuming you recorded your macro into register   a)

Next: Get a better understanding of recursive macros in Vim.