Monday, April 22, 2013

Sort Me A Column

Need to sort a column within your table without messing with the other columns?
Vim has your back with blockwise visual selections and a bit of gymnastics.

Sample table:
1 this 1 apple    is 1
2 this 4 durian   is 2
3 this 3 carrot   is 3
4 this 2 banana   is 4
5 this 5 eggplant is 5

Desired result:
1 this 1 apple    is 1
2 this 2 banana   is 2
3 this 3 carrot   is 3
4 this 4 durian   is 4
5 this 5 eggplant is 5

[wrong] Result of naive :sort command:
1 this 1 apple    is 1
2 this 4 durian   is 2
3 this 3 carrot   is 3
4 this 2 banana   is 4
5 this 5 eggplant is 5

[wrong] Result of :sort /^. / command (to skip leading numbers):
1 this 1 apple    is 1
4 this 2 banana   is 4
3 this 3 carrot   is 3
2 this 4 durian   is 2
5 this 5 eggplant is 5

A Solution

  1. Visually select the column of interest with ctrl-v
  2. Cut it with x
  3. Open a temporary scratch buffer with :enew
  4. Paste with p
  5. Sort with :sort
  6. If you need to, ensure your cursor is line 1, col 1 with gg0
  7. Visually select and yank everything, blockwisely with ctrl-vG$y
  8. Switch back to where you came from with ctrl-6 (ctrl-^)
  9. Jump to the start of your original cut with `[
  10. Paste with P

Saturday, April 20, 2013

Partial Explosion

Vim has a powerful regex spell, :help /\%[] . While it might look like the emoticon of a lunging crocodile, this little piece of regex lets vimmers match words in either their full form or any partial reduction thereof. That’s a messy explanation. A showing will help:

The pattern:

/r\%[ead]

Will match: r, re, rea, and read

Unicorns, I know!

This actually gets a lot of love in Vim’s syntax highlighting file for its own language, VimL. I wish it didn’t to be honest. I actually deplore the availability of partial keyword forms like fu instead of the full form function. But that’s not why we’re here today, so let’s move on.

Not all regex engines support this awesome atom. In fact… I don’t know of any other that does.

I am in the process of creating a vim.lang file for source-highlighter. As you would expect, it has a regex based DSL for specifying syntax items. One such item is for keywords. Easy!, I thought, I’ll just grab the keywords from Vim’s syntax files and… oh, crap… It’s infested with \%[]

So… Let’s explode them.

With a quick regex, I got the Vim keywords split out onto separate lines, like this:

a
arga[dd]
ar[gs]
bar
bn[ext]
breaka[dd]

And then we can explode out the partial variations with this little regex:

:%s/^\(.\{-}\)\[\(.\{-}\)\]/\=join(map(range(len(submatch(2))+1),  'submatch(1).strpart(submatch(2), 0, v:val)'), ' ')/

Producing:

a
arga argad argadd
ar arg args
bar
bn bne bnex bnext
breaka breakad breakadd

And just in time for dinner, too.

I'd Tap That

The Ruby world and many others have a tap() method on a class way up in the hierarchy near God that lets the bug hunting developer peek inside of objects during execution to see what the gnomes are getting up to under the covers. I needed just such a tool for my latest dabblings in Vim, so I built one and thought I’d share it here:


function! Tap(thing)
  echom string(a:thing)
  return a:thing
endfunction

It’s not very intimidating, I know, but thats a little gem of a function. I used it to debug the setting of the includeexpr option in Vim, as shown here:


set includeexpr=Tap(substitute(Tap(v:fname),'\\${\\?\\(\\w\\+\\)}\\?','\\=expand(\"$\".submatch(1))','g'))

I was suspicious that the v:fname variable was not being set properly before includeexpr was being evaluated by Vim. Tap() proved that to be the case which allowed me to focus my debug efforts on the real cause, instead of continuing to waste time fretting over the search and replace patterns in the substitute and whether I’d escaped them correctly or not. I only wish the Tap() inspiration had come to me sooner than it really did. Oh well… with Tap() as a permanent fixture in my ~/.vimrc, hopefully it won’t take me as long to think of it the next time I need its services.

Saturday, April 6, 2013

Bisectly

wtf?!

We've all been there before; all too often, in fact. There you are in the middle of a serious edit when all of a sudden some vile little gremlin spits at you from deep within Vim. "Argh!" You instantly seethe with indignant rage, vowing to righteously lance that festering pustule...! when the terrifying realisation strikes you - you don't know where this bug is coming from. "Gack." You say as you think to yourself: "It must be one of the plugins... surely? Ok... it could be in my ~/.vimrc... But no! I watered it only yesterday and there were hardly any weeds there. So, it's a plugin... But... which one? How am I supposed to find the menacing little urchin among all those other well behaving citizens in my plugin pool?" Historically the Vimmer faced with this situation has had very little option but to manually move plugins aside until the faulty one is found. This is a laborious, boring and very frustrating task. Even if you knew well enough to use binary search to speed up the process, it's still not something anyone wants to do. Thankfully now, you don't have to. Now, you can do it with Bisectly.


Bisectly is a plugin-manager agnostic fault localisation tool for finding which plugin is causing you nose-bleeds.


Bisectly uses BSFL (Binary Search Fault Localisation) to quickly whittle down the set of loaded plugins in search of the one causing you pain. It uses a cutesy command interface to identify which sessions the user considers fault-free (:Unicorns) or faulty (:Zombies). This process continues until a single plugin remains, which Bisectly considers to be the guilty party.

Plugin Manager Agnostic

Bisectly interrogates your normal Vim for its final set of &runtime paths before juggling those in the BSFL algorithm. As such, it should be completely independent of any plugin manager. I have tested it with pathogen only, but I don't foresee any issues with Vam or Vundle. Feedback appreciated on this.

Possible Futures

My original idea for the Zombies Vs Unicorns theme was actually in regard to using BSFL on locating faults within a user's ~/.vimrc file. I have a plugin that is all-but-finished for this purpose but it has been delayed until a better vimscript parser can be completed. Work on that is currently in progress.

I have dabbled with various solutions to this problem over the last year or so. Most of the other solutions were tied to pathogen or were broader, more generic endeavours (stretching beyond the realm of Vim). All of those solutions, though, utilised an automated testing framework to very rapidly locate the faulty component - in a matter of seconds. I had the idea for this incarnation of Bisectly today and rushed out the simpler manual code while it was fresh in my mind. However, I do intend to revisit this plugin to add support for an automated test framework.

Saturday, March 16, 2013

The Path of a Master VimLer


An apprentice approached a Master and said,
“Show me your best Vim.”
to which the Master responded,
“Everything in my Vim is best.
You cannot find here any piece of Vim that is not the best.”
At these words, the apprentice became enlightened.

What might the path to VimL mastery look like?
Here's what Raimondi & I think at the moment:

Trainee → Novice → Worker → Professional → Expert
  1. Trainee
    • Read the following parts of :help cmdline.txt
      • 3. Ex command-lines :help cmdline-lines
      • 4. Ex command-line ranges :help cmdline-ranges
      • 6. Ex special characters :help cmdline-special
    • Read the following parts of :help options.txt
      • 1. Setting options :help set-option
      • The following specific options:
        • :help 'debug'
        • :help 'eventignore'
        • :help 'ignorecase'
        • :help 'magic'
        • :help 'maxfuncdepth'
        • :help 'runtimepath'
        • :help 'verbose'
    • Read all of :help pattern.txt
    • Read the following parts of :help eval.txt
      • 1. Variables :help variables
      • 2. Expression syntax (skim) :help expression-syntax
      • 3. Internal variable :help internal-variables
      • 5. Defining functions :help user-functions
      • 7. Commands :help expression-commands
      • 9. Examples :help eval-examples
    • Read sections 41.1 - 41.8 in :help usr_41.txt
    • Read :help function-list (Vim’s built-in VimL library)
    • Read http://www.ibm.com/developerworks/linux/library/l-vim-script-1/index.html
    • Read http://learnvimscriptthehardway.stevelosh.com/
    • Pass the Trainee assessments
  2. Novice
    • Contribute bug fixes and small enhancements to existing plugins.
    • Read :help map.txt
      • Read :help 'timeout' option
      • Read :help 'maxmapdepth' option
    • Read :help usr_40.txt
    • Read :help autocmd.txt
    • Read :help filetype.txt
    • Read :help various.txt
      • In regard to :normal, read :help motion.txt
    • Pass the Novice assessments
  3. Worker
    • Read :help eval.txt
    • Read :help usr_41.txt
    • Create three or more plugins under the supervision of a Professional or Expert
    • Assist Novices
  4. Professional
    • Actively support five or more peer-reviewed, fully usr_41 compliant plugins
    • Thoroughly document all supported plugins
    • Supervise Workers
    • Assist in the development of reference & resource materials
    • Participate in discussions about best practice for VimL development
  5. Expert
    • Create new tools and libraries for VimL development
    • Create new/interesting/engaging/fun reference material or tutorials for an aspect of Vim/VimL
    • Mentor, guide and train other VimLers
    • Maintain a regular presence on Stack Overflow, the vim-dev mailing lists, #vim or #viml as an authority and guide on advanced Vim and VimL topics

A Plan for Assessing VimL Skills

A central registry of available assessments:
A master github repo called https://github.com/dahu/VimLAssessments.git that contains a list of VimLAss_<level_name>_<assessment_number> github repos in the various skill levels that act as assessments.
E.g.:
  1. Trainee:

  1. Novice:
    • VimLAss_Novice_010 — usr_41 compliance: use <Plug> maps

Creating New VimLAss Assessment Repositories

To create a new VimLAss assessment piece, just copy the VimLAss_Skeleton repository which has the following layout:
  • README.md — an overview of the assessment and a detailed set of tasks to be completed
  • test/<task>.vim — one or more test files per task as listed in the README.md
  • plugin directories with files as necessary. e.g.:
    • autoload/
    • doc/
    • colors/
    • compiler/
    • ftdetect/
    • ftplugin/
    • indent/
    • plugin/
    • syntax/
All VimLAss repositories are to use the https://github.com/vim-scripts/runVimTests.git unit testing framework. The repositories should contain tests for expected behaviour that initially fail. Corresponding tasks in the READEME.md file detail the necessary fixes and enhancements that the vimmler is supposed to implement to get the tests to pass. Successful completion of the assessment begins with having all tests pass. The vimmler should then apply to a Professional/Expert vimmler to have his work reviewed, gain feedback and get his assessment signed off. Actual signoff could involve digital signatures, if that path was ever deemed necessary, which is unlikely.

Even though I have provided some links to various pieces mentioned above, all of this is still very much in thought space at the moment.

I am sure we haven’t covered all bases yet. Do you have anything to contribute?

Saturday, February 9, 2013

SinTax Overdue

Syntax highlighting in Vim sucks.

I know. I haven't done as much of it as you have, but I've heard your cries from my cubicle, your desperate pleas for compassion whispered over and again at your desk to an uncaring Vim, your brave face masking knots of terror while chatting at the water cooler, and your wails of despair in the toilet stall when you thought everyone else had gone. It's been hard watching you go through this and for long now I've wished there was something I could do to ease your burden, to still your trembling hand, wipe the tears from your keyboard and tell you that "It's going to be okay." Of course, I couldn't do that. Nobody could. I'd be lying and you know it. My words would come across as naive, empty platitudes at best or, at worst, taken for cruel taunts to mock your effort, halt your progress and rob you of your due standing among the Nerds of Vim.

That was how we all lived. That is... before THEY came!



SinTax is a Vim plugin that provides a DSL for crafting syntax highlighting files.

It takes this:


" Vim syntax plugin for filetype name.
" Maintainer: Barry Arthur <barry.arthur@gmail.com>
" Israel Chauca F. <israelchauca@gmail.com>
" Version: 0.1
" Description: Long description.
" Last Change: 2013-02-01
" License: Vim License (see :help license)
" Location: syntax/vrs.vim
" Website: https://github.com/Raimondi/vrs

name    vrs
case    ignore
spell   default

keyword vrsTodo    .Todo : TODO FIXME XXX

partial token \S\+\s\+

partial separator
    \%(
      \\}          # an_escaped_\}
    \|             # or
      [^}]         # anything_but_a_}
    \)

match vrsNameErr   .Error      contained : ^\%{token}

match vrsName      .Identifier contained : ^\w\+\s\+

match vrsFlavorErr .Error      contained
  \%(
    ^\%{token}     # if_the_line_starts_with_a_token
  \)\@<=           #   before
  \%{token}        # a_token

match vrsFlavor    .Type       contained
  \%(
    ^\%{token}     # if_the_line_starts_with_a_token
  \)\@<=           #   before
  \%{vrsName}      # a_vrsName

match vrsCompItem  .Normal     contained
    \w\+           # a_word
  \|               # or
    \d\+           # a_number
  \|               # or
    ,\@<=          # a_separator_if_proceeded_by_a_comma
    \%{separator}\+

" match a composition atom: \%{pattern-name,count,separator}
match vrsCompose   .PreProc    contained contains=vrsCompItem
  \\%{             # a_literal_\{
    \S\+           # a_word_(pattern-name)
    ,              # a_comma
    \d\+           # a_number_(count)
    ,              # a_comma
    \%{separator}* # an_optional_separator
  }                # and_a_literal_}

match vrsRegExp    .String     contains=vrsCompose contained
  \%(
    ^\%{token,2}   # if_the_line_starts_with_two_tokens
  \)\@<=           #   before
  .*               # anything

match vrsCommand              contains=vrsName,vrsFlavor,vrsNameErr,vrsFlavorErr,vrsRegExp,vrsComment
  ^\%{token,2}\S.* # a_line_with_three_'tokens'_minimum

match vrsContinued .String     contains=vrsComment
  ^\s\+\S.*        # a_line_with_leading_whitespace_and_a_'token'

match vrsComment   .Comment    containedin=ALL contains=vrsTodo
  \%(              # as_long_as
    \%(
      \\ \\        #   (ignoring_any_number_of_double_backslashes_(\\))
    \)*
    \\             #   a_backslash
  \)\@<!           # doesn't_precede
  #.*$             # a_literal_#_followed_by_anything

match vrsError     .Error
  ^                # any_line_starting_with
  [^a-zA-Z0-9_#\ ] # anything_that's_not_alphanumeric,_underscore,_hash_or_a_space
  .*               # and_anything_afterwards



And generates this:



" Vim syntax plugin for filetype name.
" Maintainer: Barry Arthur <barry.arthur@gmail.com>
" Israel Chauca F. <israelchauca@gmail.com>
" Version: 0.1
" Description: Long description.
" Last Change: 2013-02-01
" License: Vim License (see :help license)
" Location: syntax/vrs.vim
" Website: https://github.com/Raimondi/vrs

" Quit when a (custom) syntax file was already loaded
if exists("b:current_syntax")
  finish
endif

" Allow use of line continuation.
let s:save_cpo = &cpo
set cpo&vim

syntax case ignore
syntax spell default
syntax keyword vrsTodo  TODO FIXME XXX 
syntax match vrsNameErr /^\S\+\s\+/ contained 
syntax match vrsName /^\w\+\s\+/ contained 
syntax match vrsFlavorErr /\%(^\S\+\s\+\)\@<=\S\+\s\+/ contained
syntax match vrsFlavor /\%(^\S\+\s\+\)\@<=^\w\+\s\+/ contained
" match a composition atom: \%{pattern-name,count,separator}
syntax match vrsCompItem /\w\+\|\d\+\|,\@<=\%(\\}\|[^}]\)\+/ contained
syntax match vrsCompose /\\%{\S\+,\d\+,\%(\\}\|[^}]\)*}/ contained contains=vrsCompItem
syntax match vrsRegExp /\%(^\S\+\s\+\S\+\s\+\)\@<=.*/ contains=vrsCompose contained
syntax match vrsCommand /^\S\+\s\+\S\+\s\+\S.*/ contains=vrsName,vrsFlavor,vrsNameErr,vrsFlavorErr,vrsRegExp,vrsComment
syntax match vrsContinued /^\s\+\S.*/ contains=vrsComment
syntax match vrsComment /\%(\%(\\\\\)*\\\)\@<!#.*$/ containedin=ALL contains=vrsTodo
syntax match vrsError /^[^a-zA-Z0-9_#\ ].*/ 
syntax match vrsError /foo/ 

hi def link vrsTodo Todo
hi def link vrsNameErr Error
hi def link vrsName Identifier
hi def link vrsFlavorErr Error
hi def link vrsFlavor Type
hi def link vrsCompItem Normal
hi def link vrsCompose PreProc
hi def link vrsRegExp String
hi def link vrsContinued String
hi def link vrsComment Comment
hi def link vrsError Error
hi def link vrsError Error

let b:current_syntax = "vrs"

let &cpo = s:save_cpo
unlet s:save_cpo

" vim: set sw=2 sts=2 et fdm=marker:



Now I feel more equipped to take your hand and say, "there, there."

Depends On: Raimondi's VimRegStyle
Coming Soon! SinTax is still in development but is ready for user testing. Feedback welcome.

Saturday, February 2, 2013

Evimosaurus Rex

E-Rex

because everyone deserves extended regular expressions

A couple posts back I compared some idiomatic Ruby with equivalent Vim forms. One of those was the multiline, whitespace-insensitive extended regular expression mode that the /x flag gives the PCRE wielder. I showed how this might be achieved in Vim as:

let PHONE_NUMBER_PATTERN = substitute(substitute('
      \ ^
      \ \%(
      \   \(\d\)           # prefix_digit
      \   [\ \-\.]\?       # optional_separator
      \ \)\?
      \ \%(
      \   (\?\(\d\{3}\))\? # area_code
      \ [\ \-\.]           # separator
      \ \)\?
      \ \(\d\{3}\)         # trunk
      \ [\ \-\.]           # separator
      \ \(\d\{4}\)         # line
      \ \%(:\ \?x\?        # optional_space_or_x
      \   \(\d\+\)         # extension
      \ \)\?
      \ $', '# \S\+', '', 'g'), '\\\@<! ', '', 'g')

echo string(PHONE_NUMBER_PATTERN)
let a_phone_number = '1-234-567-0987:1234'

echo matchlist(a_phone_number, PHONE_NUMBER_PATTERN)

Which is all good and well, but wouldn’t it be nicer if we could just do this instead:

let PHONE_NUMBER_PATTERN = ERex.parse('
      \ ^
      \ \%(
      \   \(\d\)           # prefix_digit
      \   [\ \-\.]\?       # optional_separator
      \ \)\?
      \ \%(
      \   (\?\(\d\{3}\))\? # area_code
      \ [\ \-\.]           # separator
      \ \)\?
      \ \(\d\{3}\)         # trunk
      \ [\ \-\.]           # separator
      \ \(\d\{4}\)         # line
      \ \%(:\ \?x\?        # optional_space_or_x
      \   \(\d\+\)         # extension
      \ \)\?
      \ $')

echo string(PHONE_NUMBER_PATTERN)
let a_phone_number = '1-234-567-0987:1234'

echo matchlist(a_phone_number, PHONE_NUMBER_PATTERN)

Where the ERex object was just magically waiting in the background for all of our extended regex needs…?

Install https://github.com/Raimondi/VimRegStyle and you can!


“There is no charge for awesomeness.”
— Po

This just in! the awesomeness of Extended Regular Expressions has just been added to vimple.