Why Not Find?

Why Not Find?

After posting FindInCode, someone legitimately asks: “Why not just use Find?”

Good question! Find is a Ruby supplied module to recursively search directories. So why not use it instead of Rake. Then your script won’t have a dependency on Rake.

Well, in short:

1. Rake provides two functions, recursive directory searching and line by line searching (note that egrep is a FileList method provided by Rake, and is not the same as the grep method provide by Ruby arrays). Find only handles the directory search thing.
2. I almost never use a Ruby installation without Rake installed as well. So the extra dependency is not a concern to me.
3. Since I use Rake all the time, the syntax is at my fingertips. I don’t use Find often enough to use it without checking RI.
4. The Find version of the script is several times longer than the Rake version (included below for comparison).

However …

The find version has one advantage over the Rake version in that it does not need to pull in the entire list of files into memory at once. That may be important to you.

Comparisons

Here’s the original version again:

  require 'rake'
  FileList["**/*.rb"].egrep(Regexp.new(ARGV.first))

And here is the Find version:

  require 'find'
  RE = Regexp.new(ARGV[0])
  Find.find('.') do |fn|
    next if fn =~ /(^(\.svn|CVS)$)/
    next unless fn =~ /\.rb$/
    open(fn) do |file|
      lines = 0
      file.each do |line|
        lines += 1
        puts "#{fn}:#{lines}:#{line}" if line =~ RE
      end
    end
  end

Update (31/Aug/06)

James Edward Grey II points out that my find example could use a little makeover. First, the regex that causes .svn directories to be skipped is incorrect, it only skips the actual directory, not the contents of the directory. Removing the ”$” from the match will fix that, but he provides a better solution. Replace that entire line with:

    Find.prune if fn =~ /(^(\.svn|CVS)$)/ 

I totally forgot about prune.

A minor correction is the use of the lines counter. He suggests either using each_with_index (I should have thought of that), or better yet, use file.lineno.

Thanks James!

I’ve been exploring using Rake as a Ruby Library.

Find In Code

Lately, I’ve been experimenting with using Rake as a normal library in regular Ruby scripts (rather than in Rakefiles). It turns out there are some very useful things you can do.

For example, to locate a particular piece of code in a not so small Ruby project, I often use the following command line:

    $ find . -name '*.rb' | xargs grep -ni "snippet of code" 

This quickly searches all my Ruby files and displays matching lines with file names and line numbers. However, the line is long to type and won’t work in windows (unless you have cygwin installed).

So I wrote the following Ruby program and saved it in a file called “fic” (Find In Code):

  #!/usr/bin/env ruby
  require 'rake'
  FileList["**/*.rb"].egrep(Regexp.new(ARGV.first))

An Enhanced Version

This proved so useful, that I enhanced the code to allow for searching for files ending in arbitrary extendsions and to handle more matching options on the command line.

Here’s the enhanced version:

  #!/usr/bin/env ruby
  # -*- ruby -*-

  exts = ['.rb']
  if ARGV[0] =~ /^\.[a-zA-Z0-9]+$/
    exts = []
    while ARGV[0] =~ /^\.[a-zA-Z0-9]+$/
      exts << ARGV.shift
    end
  end

  ext = "{" + exts.join(',') + "}" 

  if ARGV.size < 1
    puts "Usage: #{File.basename($0)} [.ext ...] pattern ..." 
    exit 1
  end

  require 'rake'
  FileList["**/*#{ext}"].egrep(Regexp.new(ARGV.join('|')))

And if you are using Emacs …

If you are using Emacs, and you have my “visit source” code loaded, all you need to do is put the cursor on the line you are interested in and press F2.

Oh, you would like to see the elisp code for “visit source”? Let’s see, where did I put that?

  $ cd .elisp/
  $ fic .el jw-visit-source
  ini/ini-cust.el:131:(defun jw-visit-source ()
  ini/ini-zkeys.el:7:(global-set-key [f2] 'jw-visit-source)    

Now, I move the cursor to the first line and press F2, and I find:

  ;;; Source File Visiting =============================================

  (defun jw-current-line ()
    "Return the current line." 
    (let
        ((bol (save-excursion (beginning-of-line)(point)))
         (eol (save-excursion (end-of-line)(point))))
      (buffer-substring bol eol) ))

  (defun jw-extract-file-lines (line)
    "Extract a list of file/line pairs from the given line of text." 
    (let*
        ((unix_fn "[^ \t\n\r\"'([<{]+")
         (dos_fn  "[a-zA-Z]:[^\t\n\r\"'([<{]+")
         (flre (concat "\\(" unix_fn "\\|" dos_fn "\\):\\([0-9]+\\)"))
         (start nil)
         (result nil))
      (while (string-match flre line start)
        (setq start (match-end 0))
        (setq result
              (cons (list 
                     (substring line (match-beginning 1) (match-end 1))
                     (string-to-int (substring line (match-beginning 2) (match-end 2))))
                    result)))
      result))

  (defun jw-select-file-line (candidates)
    "Select a file/line candidate that references an existing file." 
    (cond ((null candidates) nil)
          ((file-readable-p (caar candidates)) (car candidates))
          (t (jw-select-file-line (cdr candidates))) ))

  (defun jw-visit-source ()
    "If the current line contains text like '../src/program.rb:34', visit 
  that file in the other window and position point on that line." 
    (interactive)
    (let* ((line (jw-current-line))
           (candidates (jw-extract-file-lines line))
           (file-line (jw-select-file-line candidates)))
      (cond (file-line
             (find-file-other-window (car file-line))
             (goto-line (cadr file-line)) )
            (t 
             (error "No source location on line.")) )))

Uncle Bob has a couple examples of why performance tuning almost always has surprises in store for you.

Not What You Expect!

Bob Martin (Uncle Bob) relates a couple of stories on performance tuning and why your expectations are nearly always wrong.

The moral of the story … measure, measure, measure whenever you have a performance problem. The root cause is prabably not where you expect it to be.

Rails Edge

This is exciting. I have been wanting to talk about this for some time. Well, the time has finally come and I’m happy to announce that the Pragmatic Programmers are putting together a series of regional conferences called The Rails Edge.

The conference will be a single track, so all the speakers at the conference will be available for interaction at all the talks, not just during their own presentation. The speaker selection is great (I feel humbled to be a member of the group). We are hoping that it will a dynamic experiance with the speakers interacting, not only with the audience, but with the other members of the staff.

So, if you didn’t make it into the “Lucky 250” club going to RubyConf this year, here is another chance to meet and greet Rubyists in your area.

I hope to see you there.

Update: You can read the official Pragmatic Programmer’s Announcment too.

The Ruse wiki software gets some updates.

Some Changes to Ruse

I’ve been really pleased with the way Ruse has been handling wiki related spam on the Ruby Garden site. Now that Ruse has been deployed for a while, I’ve tweaked a few things to make it even easier to clean up spam.

• Queued Review Mode: The original RubyGarden wiki took an annoying number of clicks to clean a page of spam. Ruse has the cleanup down to one click, but it takes an additional two clicks to get to the next page to review. (goto to review page, choose next page to review). The new Queued Review mode will now automatically advance to the next page for reviewing, bringing you to a true “one click per page” review mode. Click on the “Work from Queue” link on the review page.

• Bug Fix: The review page now correctly displays the differences for the version you are reviewing. The original behavior incorrectly showed the wrong revisions in the differences section (although the page content was correct).

• Minor changes and fixes: The size of the log page is now configurable (rather than the fixed 25 entry size). Demotion during review is now properly logged.

Future Directions

One technique that has proved very useful in preventing spam is the content filtering that Ruse is able to do. We download the “bad url” list from chonqged.org nightly and will reject all posts URL that match that list. Currently, we can add additional URLS to the blacklist by editting a configuration file on the server. Future versions of Ruse might add a web interface to the local blacklist.

We had some fireworks on the 4th of July … Unfortunately they weren’t the usual kind.

Fireworks on the 4th

It is a tradition in the US to fire off fireworks on the 4th of July in celebration of Independence day. Well, the Weirich household had some fireworks, but it’s not what you would expect.

What we thought at the time was a really loud firecracker from the neighbors turned out to be a close lightening strike, probably hitting the phone line. I discovered later that my DSL router (attached to the phone line) was dead (as in no lights, no power).

After a picking up a new DSL router on Wednesday from the phone company, it still didn’t connect and the phone company promised to send a technician out one Friday. Arrgh! Three days with no internect! Sigh.

The technician arrives Friday, checks out the wiring and decides to upgrade the “system”. Part of the upgrade is another new DSL router (different model from the replacement unit I picked up Wednesday). We connect up the laptop to the DSL router and everything looks great.

So once the phone guy leaves, I start hooking up the rest of the network: firewall router, network hub, wireless base station, and the other computers in the house.

And nothing works.

It looks like the lightening not only took out my DSL router, but it also got the firewall, network hub and wireless base station. Fortunately, I have another firewall and hub in my spare parts box.

Bad Cables?

I’m not certain of the total extent of the damage yet. The cables running to the other computers in the basement seem to be bad now (tested by against my working laptop). Since I don’t have long enough replacement cables, I haven’t checked out the network cards in two of the computers yet.

I am a bit surprised by the cables going bad. I understand delicate electronic equipment fried by lightening, but the cables are just wires and connectors. For them to be bad must mean the strike was strong enough to short them out somehow. And that doesn’t sound good for the network cards at the other end of the cable. Sigh.

On the good side, the my desktop computer that sits right next to the network equipment is up and running on the network. So one would that the network card in it is ok. However, it does seem to be a bit slow when browsing web pages. I mean really slow. I timed it against my laptop. What loads two seconds on my laptop takes over 20 seconds on the desktop. But that sounds more like a network configuration error than a hardware issue. Sigh, more work to do.

At any rate, it looks like there will be a run to the local computer store soon.

Several people have been having RubyGems issues.

Deleting Your RubyGems Cache

Several people have been reporting problems where RubyGems doesn’t find a gem on RubyForge, or gives other strange errors. It seems there was a gem on RubyForge that gave the Gem indexing software some headaches. And as a result, the gem index was corrupted. If you downloaded the corrupt index, then you may be experiencing strange problems as well.

The good news is that the problem is easy ot fix. Just delete the gem index file. You can find it in the directory reported by the “gem env gemdir” command. For example:

$ gem env gemdir
PATH_TO_DEFAULT_GEM_REPOSITORY
$ rm PATH_TO_DEFAULT_GEM_REPOSITORY/souce_cache

Where PATH_TO_DEFAULT_GEM_REPOSITORY is your default gem repository.

If you run on a Unix system, you default gem repository is probably non-writable from your regular user account. In that case, you will need to use “sudo” (or its equivalent) on the “rm” command. You will probably also have a secondary writable cache in your .gem directory that you will want to delete.

$ rm $HOME/.gem/source_cache

The next time you run gems, will will refresh the cache by downloading the index from scratch.

We Need Proposals

Write Up Your RubyConf 2006 Talk Proposal

David A. Black is asking for proposals for RubyConf 2006. If you have an idea for a talk, please feel free to propose it for the conference (and you can use this page). The deadline is June 30th, so don’t delay.

RubyConf has always been one of my favorite conferences to attend. If you are thinking going, then consider this my personal invitation to attend. I hope to see you there.

When I checked the wiki last night, I found that over 100 pages had been updated.

Not Spam, Just Weird Stuff

So I started checking out exactly what had changed. It seems that almost all changes were by a single anonymous individual. Over the coourse of 7 hours he touched over 100 pages and reverted them to the earliest possible version. What’s up with that?

It may not be spam, but it certainly counts as destructive behavior. So, I threw him in the tarpit and all his changes into the tarit. In 20 seconds I wiped out his seven hours of hard work.

I hope he had a good day. I certainly did.

New Ruse Logo

And in other news, we have a new Ruse Wiki logo:

There seems to be some confusion.

Spotted on the Rails mailing list:

Re: RailsConf in London
So? Is Jim Weirich coming or not?
YES: http://europe.railsconf.org/articles/2006/05/03/announcing-railsconf-europe
NO: http://skillsmatter.com/menu/255

The answer is YES, I will be there. I was a late addition to the list and my name might not have made it into the announcement from Skills Matter.