Closures in VimL

---

I’ve talked about Anonymous Functions in VimL before, but what about closures?

What is necessary for a language to be able to say that it supports closures? What is sufficient for someone to be able to fake their way through it?

Here is the first example from Wikipedia’s Closure article:

def start(x): def increment(y): return x+y return increment first_inc = start(0) second_inc = start(8) first_inc(3) # returns 3 second_inc(3) # returns 11 # The x value remains the same for new calls to the function: first_inc(1) # returns 1 second_inc(2) # returns 10

Here is a faked-up alternative in VimL:

function! Start(x) let obj = {} let obj.x = a:x func obj.increment(y) return self.x + a:y endfunc return obj endfunction

let first_inc = Start(0) let second_inc = Start(8) echo first_inc.increment(3) echo second_inc.increment(3) echo first_inc.increment(1) echo second_inc.increment(2)

Okay, so not really what you think of as a closure, but it lets you get similar things done. Wikipedia says:

Closures are typically implemented with a special data structure that contains a pointer to the function code, plus a representation of the function’s lexical environment (i.e., the set of available variables) at the time when the closure was created.

— Wikipedia

The manual marshalling in our object-oriented implementation above probably doesn’t count, but it’s a similar idea… isn’t it?

Vim certainly has First Class Functions and with VimaholicsAnonymous, it has Anonymous Functions… Does it have closures? If not… does it matter? Can you achieve something similar using its prototypal object notation?

Welcome to the Search Party

Enhanced searches, highlights and matches for Vim

Raimondi and I wrote a nifty little plugin and, well, we’d like to invite you to join the SearchParty.

SearchParty has these awesome features:

Visual Searches

• * Searches for the next occurrence of the currently selected visual text.
  
• # Searches for the prior occurrence of the currently selected visual text.
  
• & Starts a :substitute using the currently selected visual text.
  

I’ve had a goodly amount of <3 on #vim for the visual & command.

Literal Search

<leader>/ prompts for a literal string to search for. This does NOT use a regular expression, so the characters you type here will be searched for literally without any magic interpretation. The <Up> key scrolls through the prior literal search history.

This one is a real crowd pleaser. People come for the highlight candy, but they stay for Literal Search.

Ranged Search

:7,12 RSearch foo

Searches for "foo" only within the range from lines 7 through 12, both inclusive. The default range is % (the whole buffer).

The normal next and previous keys (n and N) cycle within the range.

Multiple Replacements

<leader>mp prompts for a Search term and then prompts for space separated Replacement terms (use \\ to escape desired spaces). The current line is then duplicated as many times as there are replacements, minus one, and the Search term is then replaced on each line with each successive Replacement.

Perhaps an example would far better explain this:

Given the line:

   Don't credit it on the sunshine

With the cursor on that line, the command:

<leader>mp

Followed by:

Search:credit
Replace:blame

Will produce:

   Don't blame it on the sunshine

That’s no better than :s/// I know, but with the cursor still on the same line, check it:

<leader>mp

Followed by:

Search:sunshine
Replace:moonlight good\ times

Will produce:

   Don't blame it on the sunshine
   Don't blame it on the moonlight
   Don't blame it on the good times

Blaming it on the boogie is left as an exercise for the reader.

Search Highlighting

IIRC, this is where SearchParty all began — the ability to easily highlight the word under the cursor without silly machinations like: *#

• <C-L> Temporarily clears search highlight.
  
• <C-BSlash> Toggles search highlighting.
  
• <leader>* Highlights all occurrences of word under the cursor.
  
• <leader>mah Toggle automatic highlight of all occurrences of word under cursor.
  
• <leader>g* Highlights all occurrences of WORD under the cursor.
  

You might think that <leader>mah is a bit verbose but toggling automatic word highlighting is not something I think you’ll need to do very often. If it does bother you, though, this mapping and all of the mappings in SearchParty are <Plug> maps, so you can customise them to your own tastes. Instructions for doing so are in the plugin docs.

Highlighting Print Command

Modern grep commands highlight the search term within the resulting lines (if you so desire). With SearchParty, Vim’s :g// command does too now:

:g/something/P

Will show the matching lines with all occurrences of "something" on those lines highlighted.

This command can also be used for an arbitrary range and it will highlight the most recent search pattern (@/) within those lines. E.g.: :10,20P

Set Search

Sometimes you’d like to highlight that word over there without having to go there, use <leader>* on it, and come back. <leader>ms is the answer.

Matches

This is one of my personal favourites — the ability to have up to six different strings highlighted in big, bold, bright colours all across the screen wherever they appear. The <leader>mm command prompts you for a string (kindly inserting the current word for you) to highlight. If you go past six then the first one is forgotten and replaced with your latest string (cycling like this ad infinitum). I find this useful for ensuring that a few particularly important strings don’t escape my attention throughout a document.

M.A.S.H.

Lastly, we have the Motion Activated Search Highlighter: when you press n/N/#/*/g#/g*, it highlights the match under the cursor differently to all the other matches on screen (if you have :set hlsearch activated). If you don’t use hlsearch, then it will still highlight the current match.

I really like this feature. I used to find it difficult to see where my cursor was when it was within a highlighted search term — the other similarly highlighted blobs would all compete for my attention. M.A.S.H makes this a no-brainer now.

SearchParty

So, welcome to the party. Grab yourself a buffer and get highlighting, make some matches or MASH out on the lounge. Enjoy your evening.

Y u no, Vim?

Well, Y you can, it would seem…

I was recently reminded (hi, dhruvasagar) of the Y combinator while idling on #vim and I thought to myself… can you Y, Vim? I decided to find out.

I just recently wrote about Anonymous Functions in Vim and they form the cornerstone of my attempt at Y-ing here. I used Mike Vanier’s excellent article: The Y Combinator as a guide (any and all errors are wholly mine); reading it will fill in the huge holes I have chosen to skip over here.

Mr Fibonacci is in the studio with us today, representing la raison d’recurse.

Mr Fibonacci, Zero?

"=> 0

Lovely! Er… and 14?

"=> 377

Amazing!

Now to some code… First up is the standard recursive definition of Fibonacci:

function! RecursiveFibonnaci(n) if a:n == 0 return 0 elseif a:n == 1 return 1 else return RecursiveFibonnaci(a:n - 1) + RecursiveFibonnaci(a:n - 2) endif endfunction

And asking our studio guest…

echo RecursiveFibonnaci(10) "=> 55

Decent. Now just for fun, here’s the same thing written as an Anonymous Function:

let RF = Fn('(n) => if a:n == 0' \.'| return 0' \.'|elseif a:n == 1' \.'| return 1' \.'|else' \.'| return call(g:RF, [a:n - 1]) + call(g:RF, [a:n - 2])' \.'|endif')

Fn() is provided by VimaholicsAnonymous.

How does she measure up, Mr Fibonacci?

echo RF(11) "=> 89

…and does it agree with our recursive stalwart?

echo RecursiveFibonnaci(12) == RF(12) "=> 1

Standard! But we haven’t even begun to look at Y yet, so here is Mike’s AlmostFibonacci a la Vim:

function! AlmostFibonacci(f) silent! unlet b:f let b:f = a:f return Fn('(n) => if a:n == 0' \.'| return 0' \.'|elseif a:n == 1' \.'| return 1' \.'|else' \.'| return call(b:f, [a:n - 1]) + call(b:f, [a:n - 2])' \.'|endif') endfunction

You will notice that it contains an almost identical copy of the anonymous recursive Fibonacci (RF) we declared above. It’s not quite the same, of course, because this version is not meant to be self-recursive. It calls the function f provided to AlmostFibonacci. The little silent! unlet b:f dance and the use of a buffer variable in the first place is just some necessary mechanics to pass AlmostFibonacci's f argument into the inner anonymous function. Basically, we cheat through the use of a global. I use a buffer-local here as a baby-global.

How are we doing, Mr F.?

echo call(AlmostFibonacci('RecursiveFibonnaci'), [12]) "=> 144 echo RecursiveFibonnaci(12) == call(AlmostFibonacci('RecursiveFibonnaci'), [12]) "=> 1

Respect.

Heh… The astute among you are half way through a hate comment right now that I have violated one of the laws of thermodynamics, or at least common decency by employing RecursiveFibonnaci in the call to AlmostFibonacci there. Relax. I know I’m cheating. I just wanted to prove that the code worked. We’ll get to actual Y below. :-p

I hesitated whether I would even show this next piece. It’s using the exact same functions as we just used above, but I am using a slightly different call() interface. I actually use this call() interface in the real Y; I wanted you to know that this difference wasn’t.

echo call(call('AlmostFibonacci', ['RecursiveFibonnaci']), [13]) "=> 233 echo RecursiveFibonnaci(13) == call(call('AlmostFibonacci', ['RecursiveFibonnaci']), [13]) "=> 1

And here she is: Y in Vim!

function! Y(f) let b:f_ = a:f return call(a:f, [Fn('(x) => call(call("Y", [b:f_]), [a:x])')]) endfunction

Again, you will notice the same buffer-local dance for marshalling the function in a:f into the inner anonymous function.

And, hopefully unsurprising by now, here is how we define Fibonacci with Y:

function! Fibonacci(n) return call(call('Y', ['AlmostFibonacci']), [a:n]) endfunction

Still kosher, Mr F.?

echo Fibonacci(14) "=> 377 echo RecursiveFibonnaci(14) == Fibonacci(14) "=> 1

Splendid! But… how does the Y version compare to the recursive version?

let start = reltime() call RecursiveFibonnaci(20) echo reltimestr(reltime(start)) "=> 0.99 seconds

uh huh…

let start = reltime() call RF(20) echo reltimestr(reltime(start)) "=> 1.066 seconds

I guess the indirection there causes some overhead, but nothing disastrous…

let start = reltime() call Fibonacci(20) echo reltimestr(reltime(start)) "=> 14.56 seconds

:-( There is no God!

It would seem that although "possible", Y in Vim is not advised*. Or, at least, not the way I approached it. Perhaps there’s a brighter way? I’d love to see it.

(*): Y is not advised anywhere, as far as I can tell, except as a mind-stretching exercise from the church of functional dys.

Just to scare little Vimmers late at night, after running this code, the number of anonymous functions in my Vim session was 48098! This code maims unicorns.

That’s Y!

What If The Romans Had Vim?

---

This is much more about VimL than MDCLXVI strings; if this code doesn’t rape your cat, you’re welcome.

Call it a kata if you will; I decided over breakfast to have a go at writing a Roman Numeral to Arabic Number converter in VimL. It’d been sufficiently long since I’d written such a converter in any language, and I’d never done one in VimL, so I thought, why not?

I prefer to think away from the console, so I grabbed a notepad and pencil and started doodling a solution based on how I think about parsing Roman Numerals in wet-ware. Roughly, that looked a bit like:

to_arabic(str) new stack for each roman_letter in str val = arabic_value(roman_letter) if val < stack.tos() stack.push(val - pop()) elseif val == stack.tos() stack.push(val + pop()) else stack.push(val) end end return sum(stack) end

Yes, I am aware that might expose potentially embarrassing belief systems I might be operating under regarding life, the universe and everything. Meh.

I did some mental as well as paper-based run-throughs of the algorithm to convince myself that it would work and then implemented it in VimL. I don’t have the VimL solution to show you because after getting a working version I began refactoring it as I saw opportunities and generalities lurking within the code.

Here is what I ended up with:

function! Roman() let obj = {} let obj.values = {'M':1000, 'D':500, 'C':100, 'L':50, 'X':10, 'V':5, 'I':1 } func obj.to_arabic(str) dict let self.prior = 0 return eval(join(map( \ reverse(map(split(a:str, '\zs'), 'self.values[v:val]')) \, 'self.calc(v:val)'), '+')) endfunc func obj.calc(v) dict let [n, self.prior] = [(a:v < self.prior ? -a:v : a:v), a:v] return n endfunc return obj endfunction

So while my pseudo-code used a stack and processed the string in original order, the final algorithm reversed the string and just tracked the prior value each time through the loop (map()).

Some explanations of the weirder VimL:

• I’m using VimL’s Object Oriented approach which is more like javascript’s than C++'s. The Roman() function is actually an object generator. Each such object then has a to_arabic(roman_string) method.
• I’m also using VimL’s functional-ish approach to processing data lists. Reading such constructs is usually better from the inside out, which begins with:
  
  • The split(a:str, '\zs') yields a LIST OF individual LETTERS. Unfortunately, the explanation at :help /\zs doesn’t include the idiomatic use shown here of exploding strings out to a list of their component characters.
    
  • The map( LIST OF LETTERS , 'self.values[v:val]') converts individual Roman Numerals to equivalent Arabic numbers. This, therefore, returns a LIST OF NUMBERS. VimL’s map() function takes the list first and an expression to be evaluated against each element in turn (the resulting list returned by map() is the modification of each element through this evaluated expression). A simple example might help; this generates squares:
    

    echo map(range(1,10), 'v:val * v:val')

    The archaic looking `v:val` is VimL's Way of exposing access to the value of
    the element. If it helps, this would be equivalent to the Ruby code:

    (1..10).map {|val| val * val}

    map() in VimL is destructive, not that that matters here, though.

  • The map( LIST OF NUMBERS , 'self.calc(v:val)') inverts numbers in the list if they precede bigger numbers. Remember the original string order of letters is reversed in this algorithm.
    
  • The eval(join( LIST OF NUMBERS , '+')) sums the list.
    

All in all… I think this algorithm does what it’s supposed to do, and it does so with less aggression than solutions I googled for afterwards. o_O

At least the first comment in that last one tries to head the complainant in the right direction early.

The other thing I wanted to show today is micro-tests. Of course, being able to refactor sloppy code into a better solution is made practical with tests.

We all know and love testing, and Vim has decent plugins for doing it properly when you have to (i.e. writing your own plugin.) However, for quick’n'dirty jobs like this, I usually just drop a small inline test at the end of the script:

if expand('%:p') == expand('<sfile>:p') let fail = 0 for t in [ \ [1, 'I'] \, [2, 'II'] \, [3, 'III'] \, [1992, 'MCMXCII'] \, [1999, 'MCMXCIX'] \] if t[0] != Roman().to_arabic(t[1]) echo 'Fail: ' . t[0] let fail = 1 endif endfor if ! fail echo "Ok" endif endif

The if expand('%:p') == expand('<sfile>:p') trick is to allow this script to be `:source`d from another script without triggering the unit-tests. I didn’t need this here because roman numeracy has been out of fashion for a while now, but I thought I would show this here as an added bonus. I’ve shown few tests here to keep the post brief. My real tests for this little algorithm number in the fifties. I bet even then I’ve overlooked some poignant design aspect of Roman Numerals. This was meant to be an exercise in the general approach rather than aiming to provide an exhaustively accurate turn-key solution. One notable absence in that vein is the lack of data validation to ensure the roman numeral string is not of Celtic descent. For those with the inability to read between the lines (osse), that last paragraph means: if I have flubbed some VimL, I’d love to hear from you; if I’ve merely upset dead Romans and their counting system, I don’t so much care.

Update: The testing code has been extracted out into a tiny little plugin called vim-u-test if you're interested in experimenting with it.

VimL List Gymnastics

---

Some common flexing between lists and dicts in VimL.

let a_list = range(1,10) echo a_list "=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] " from list -> dict let a_dict = {} call map(copy(a_list), 'extend(a_dict, {v:val : (v:val * v:val)})') echo a_dict "=> {'1': 1, '2': 4, '3': 9, '4': 16, '5': 25, '6': 36, '7': 49, '8': 64, '9': 81, '10': 100} " from dict -> list echo keys(a_dict) "=> ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10'] echo map(keys(a_dict), 'str2nr(v:val)') "=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] echo values(a_dict) "=> [1, 4, 9, 16, 25, 36, 49, 64, 81, 100] " collecting a single dictionary from multiple dictionaries let b_text = "one thing\ntwo things\nthree more things" let b_dict = {} call map(split(b_text, '\n'), 'extend(b_dict, {split(v:val)[0] : v:val})') echo b_dict "=> {'one': 'one thing', 'two': 'two things', 'three': 'three more things'}

Anonymous Functions in VimL

Admit it, you’ve wanted this for a long time now.

Update: These two functions are now available in VimaholicsAnonymous.

VimL’s list sort() method allows the caller to provide a custom comparison function with which to sort by. Unfortunately, VimL doesn’t support anonymous functions (sometimes called lambdas), so the caller is forced to pre-write their comparator as a full-fledged function and provide its funcref to sort(). Well, that ends today. Now, VimL has anonymous function love! Check it:

This humble little snippet of code lets vimmers declare anonymous functions on the fly:

let fn_idx = 0 function! Fn(fn_form) let [fn_args, fn_body] = split(a:fn_form, '\s*=>\s*') let g:fn_idx = g:fn_idx + 1 let fname = 'AnonFn_' . g:fn_idx let b_elems = split(fn_body, '|') let b_elems[-1] = 'return ' . b_elems[-1] exe 'func! ' . fname . fn_args . "\n" \. join(b_elems, "\n") . "\n" \. 'endfunc' return function(fname) endfunction

Like this:

let x = ["one","two","three","four","five","six","seven","eight","nine","ten"] echo sort(x, Fn('(a, b) => len(a:a) > len(a:b)'))

The magic is in the Fn() call. It takes a string argument of the form:

(arguments) => function-body statements separated by | (pipe)

The last statement will be implicitly returned by the anonymous function, so no need to explicitly add a return statement.

Unicorns, or what?!

What? You want more. Certainly, sir. Behold:

function! Fx(fn, ...) return call(a:fn, a:000) endfunction

That lets you execute an anonymous function, like:

echo '2 ^ 6 = ' . string(Fx(Fn('(a, b) => pow(a:a, a:b)'), 2, 6))

Any cooler and you’d need your jumper! :-D Oh, but there’s more… I’m just getting warmed up!

Just because I like you, here’s a little extra gift:

let Mul = Fn('(a,b)=>let x = a:a | let y=a:b | x*y') echo '5 * 6 = ' . Fx(Mul, 5, 6)

That lets you call your funcref'd anonymous function by name. Did you see what I did there? Yes, I cheated God! I named an annonymous function. Cool, eh?

So, apart from the obvious, is this really that amazing? My vote is: YES! I have plans for this technology. Here’s a sneaky hint of where I’m looking to take this:

CompileMacros Mul = ((a,b) => let x = a:a | let y=a:b | x*y) echo '5 * 6 = ' . #(Mul, 5, 6) echo '2 * 6 = ' . #(((a, b) => a:a * a:b), 2, 6)

That’s right. I want hygenic macros in VimL. And that code works in my current experiments. The notation is stolen from… clojure I believe, where #(…) executes a lambda.

The call to CompileMacros will process any macros in the surrounding expressions before sourcing the result. Just for completeness, defining macros currently looks like this:

call Macro('(\+\((.\{-})\s*=>.\{-}\))', "Fn('\\1')") call Macro('\%(\n\|^\)\@<=\s*\(\w\+\)\s*=\s*\(Fn(.*=>.\{-})\)\s*\n', \ "let \\1 = \\2\n" ) call Macro('#(\(.*\))\n', "Fx(\\1)\n") call Macro('#Fn(\(.\{-}\))\n', "Fx(Fn(\\1)\n")

But that will change in the next version. Using regex to parse is an abomination; I was merely proof-of-concepting here. I’ll use VimPEG instead.

Stay tuned, if bending the spoon is what you’re in to.