Caller's name lookup

Catalog of dial plans
MikeTelis
Posts: 1581
Joined: Wed Jul 30, 2008 6:48 am

Re: Caller's name lookup

Post by MikeTelis » Sun Aug 08, 2010 4:32 am

You need to change this line:

sys.Dial("#{sys.Username}@local[fu=#{name}]",45)

sys.Username is your Sipsorcery login name which is the same as the name of your "main SIP account". Naturally, all incoming calls are forwarded to this SIP account.

It would be enough to change sys.Username to req.URI.User:

sys.Dial("#{req.URI.User}@local[fu=#{name}]",45)

You'll find more sophisticated code (separating trunk/user) in Flexible table-controlled dialplan, Version 2.

User avatar
medfordite
Posts: 62
Joined: Thu Jul 15, 2010 1:57 am
Location: Medford Oregon
Contact:

Re: Caller's name lookup

Post by medfordite » Wed Sep 01, 2010 10:25 pm

OH WOW! I Am so glad I stumbled upon this thread while being a bit bored this afternoon. :)

I used your Complexplan found on your site Mike with the CNAM feature and followed the API setup with whitepages. So, it is working pretty good now. Got the clue that my cell phone apparently says the city and state for me instead of my name so that was a good place to start with the caller ID identification.

I hardly get 200 calls daily so I am sure I will fit fine within the whole scheme of things for the whitepages API stuff.

Thanks for the great scripting for this. Hopefully now everyone will start showing up in the list when they call. I just went to Whitepages and did a few reverse searches for my curiosity of the most common callers I have and it gave me ideas as to who to filter out and add a name to them since they are unpublished.
-=-Frogs have it easy...they can eat what bugs them!-=-

MikeTelis
Posts: 1581
Joined: Wed Jul 30, 2008 6:48 am

Re: Caller's name lookup

Post by MikeTelis » Thu Sep 02, 2010 5:27 am

White Pages used to allow 1500 API requests per day; now they reduced it to 200 :-( While it's still enough for most users I'd recommend including most frequent callers in your CNAM hash table. The table is searched first thing and if your caller's number is found in there, the dialplan will not issue White Pages API search request. This way, you save on the number of API requests and make sure you never hit the limit.

johnnx
Posts: 10
Joined: Tue Nov 02, 2010 10:28 am

Re: Caller's name lookup

Post by johnnx » Thu Dec 23, 2010 11:29 pm

Hi all,

may I ask how to adjust the code for a lookup of German phone numbers?
The look-up link is "http://www4.dastelefonbuch.de/?cmd=search&kw=" with a German phone number following, e.g. "http://www4.dastelefonbuch.de/?cmd=search&kw=0311234567".
I am not quite sure how to adjust this code for receiving "missed call" emails with the caller's information as the following try didn't work:

Code: Select all

# WP_key = 'Your_API_KEY_VALUE'     # not needed for German lookup service

# CNAM Hash Table (phonebook)
CNAM = {
  # will be filled in future
}

  if sys.In               # If incoming call...
    name = req.Header.from.FromURI.User.to_s    # Get caller ID

    if !(cname = CNAM[name]) && name =~ /^1([2-9]\d\d[2-9]\d{6})$/
      url = "http://www4.dastelefonbuch.de/?cmd=search&kw=name"
      cname = $1 if sys.WebGet(url,4).to_s =~ /"displayname":(?:\[)?"([^"]+)"/
    end

    sys.Log("Caller's number: '#{name}'"); sys.Log("Caller's name = '#{cname}'") if cname

    sys.SetFromHeader(cname || name, nil, nil)  # Set FromName for sys.Dial

    # Forward call to bindings. Change FromURI when forwarding to
    # @local, or else Bria won't find contact in its phonebook!

    sys.Dial("#{sys.Username}@local[fu=#{name}]",45)

    sys.Respond(480, "#{sys.Username} Not online") # switch to voice mail
    sys.Email("my@email.address", "Missed Call", "You have a missed call from #{req.Header.From.FromURI.User}")


  else

# Outbound plan
Neither is there a lookup of the number nor do I get an email.
Does anyone have an idea? Any help appreciated. Thanks.

MikeTelis
Posts: 1581
Joined: Wed Jul 30, 2008 6:48 am

Re: Caller's name lookup

Post by MikeTelis » Fri Dec 24, 2010 7:35 am

Your mod to my code is infested with bugs. First off, it's never executed because my original code is runs WhitePages lookup only if caller's number is in US/Canada format:

... && name =~ /^1([2-9]\d\d[2-9]\d{6})$/

Apparently phone numbers in Germany do not match the pattern above.

2nd, your URL returns quite a long page and it can't be loaded using sys.WebGet because of its length. 3rd, this page in a totally different (from WhitePages' API) format and of course, regexp search on displayname can't find the data you need.

You should employ YQL to shrink page data. I wrote a sample query to help you with that. You need to try this query with various phone numbers (business, personal, etc). Probably some numbers return several names (this isn't common, but happens with WhitePages). Of course, you'll need to modify xpath so that it would return all the data you need. Finally, you'll need to parse the output to extract (and format) caller's name.

P.S. Are you sure there is no API? It would be much faster and easier to use API call(s) rather than scrap HTML page.

johnnx
Posts: 10
Joined: Tue Nov 02, 2010 10:28 am

Re: Caller's name lookup

Post by johnnx » Fri Dec 24, 2010 9:17 am

Hello Mike,

thank you for your reply.
Well, I am not an expert on that field. I tried to search how other callmonitors solve that problem, e.g. for the dsl and voip router Fritz!Box, and I found the following link:
http://www.wehavemorefun.de/fritzbox/in ... alllog.php
They are using the mentioned http://www4.dastelefonbuch.de/?cmd=search&kw=$number. So I assumed that's the way to do it..

Seems to be much more complicated than I thought though :(

I deleted

Code: Select all

&& name =~ /^1([2-9]\d\d[2-9]\d{6})$/
so hopefully everything is looked up and if the number doesn't match a German number pattern I'll rather get an error email than try to fiddle with that line now for the time being.

I inserted YQL code obviously at a wrong place or with wrong parameters as the code reading breaks up at line 41 which would actually be the first YQL line

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>

Code: Select all

  if sys.In               # If incoming call...
    name = req.Header.from.FromURI.User.to_s    # Get caller ID


    # Check CNAM first. If not found and US number, try to lookup caller's name in Whitepages
    if !(cname = CNAM[name])
<?xml version="1.0" encoding="UTF-8"?>
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng"
    yahoo:count="20" yahoo:created="2010-12-24T09:03:30Z" yahoo:lang="en-US">
    <diagnostics>
        <publiclyCallable>true</publiclyCallable>
        <url execution-time="1028" proxy="DEFAULT"><![CDATA[http://www4.dastelefonbuch.de/?cmd=search&kw=name]]></url>
        <user-time>1037</user-time>
        <service-time>1028</service-time>
        <build-version>9962</build-version>
    </diagnostics> 
</query>
      cname = $1 if sys.WebGet(url,4).to_s =~ /"displayname":(?:\[)?"([^"]+)"/
    end

    sys.Log("Caller's number: '#{name}'"); sys.Log("Caller's name = '#{cname}'") if cname

    sys.SetFromHeader(cname || name, nil, nil)  # Set FromName for sys.Dial

    # Forward call to bindings. Change FromURI when forwarding to
    # @local, or else Bria won't find contact in its phonebook!

    sys.Dial("#{sys.Username}@local[fu=#{name}]",45)

    sys.Respond(480, "#{sys.Username} Not online") # switch to voice mail
    sys.Email("it-correspondence@gmx.net", "Missed Call", "You have a missed call from #{req.Header.From.FromURI.User}")
  else

MikeTelis
Posts: 1581
Joined: Wed Jul 30, 2008 6:48 am

Re: Caller's name lookup

Post by MikeTelis » Fri Dec 24, 2010 9:22 am

Update: There are good news and bad news. Good news is that my query works on the numbers returning multiple entries. Bad news is that returned HTML may contain escape codes for umlauts (such as &auml;) and, probably, some other characters. So, you'll need to decode that, too.

I found a couple of related projects: HTML entities and HTML helpers. Of course, they are heavy and you shouldn't borrow all the code but they contain escaped char codes you'll need.

johnnx
Posts: 10
Joined: Tue Nov 02, 2010 10:28 am

Re: Caller's name lookup

Post by johnnx » Fri Dec 24, 2010 9:28 am

Thanks.
As you might guess, with my little knowledge of coding I could live with "&auml" as a text output as long as the lookup procedere would work at all :-) Besides the yql says it's in utf8 so "&auml" should produce the right output to my understanding. I'll have a few more tries before giving up on that :)

MikeTelis
Posts: 1581
Joined: Wed Jul 30, 2008 6:48 am

Re: Caller's name lookup

Post by MikeTelis » Fri Dec 24, 2010 9:42 am

If you can't tell (in Ruby) US number pattern from German one, you don't have the skills needed to write the code. Sorry, I didn't mean to offend you, it's just a matter of fact.

I should be able to write the code but I'll need your help (mostly because I don't speak German and it's difficult for me to read and understand it).

First, I will need to know the format in which you're getting caller's number from your DID provider. If you're using one of my dialplans there is a debug line:

sys.Log("** Call from #{req.Header.From} to #{req.URI.User} **")

which would output caller's number to the Console. Please give me a couple of samples so that I know which format it comes from, for example:

** Call from 0049302345678 to ...

Second, play with my YQL query (try it with various phone numbers from your contact list) and find out which umlaut and other special codes appear in there. I guess I will need to decode them to the base (umlaut A -> A).

While playing with the phone numbers, note if there are any numbers where the query would return incorrect data (no data at all, for example). If this happens, check if you can retrieve correct data from dastelefonbuch.de web page.

Give me a list of all "incorrect" numbers. If concerned about privacy, PM the numbers.

MikeTelis
Posts: 1581
Joined: Wed Jul 30, 2008 6:48 am

Re: Caller's name lookup

Post by MikeTelis » Sat Dec 25, 2010 6:51 pm

I have changed the code so that it would use Whitepages for US/Canada CNAM lookup and Dastelefonbuch.de for German numbers. Here is the code:

Code: Select all

  if sys.In               # If incoming call...
    @cid = req.Header.from.FromURI.User.to_s    # Get caller ID

    @cid.sub!(/^(\+|00|011)/,'')   # Remove international prefixes, if any
    @cid = ('1' + @cid) if @cid =~ /^[2-9]\d\d[2-9]\d{6}$/ # Prepend 10-digit numbers with "1" (US country code)
    @cid = ('49' + $1) if @cid =~ /^0((1[5-7]\d|30|40|69|89|\d\d1|\d{4})(\d{4,}))$/ # Prepend Germany country code

    prs = req.URI.User.split('.')  # parse User into chunks
    @trunk = prs[-2]               # get trunk name
    @user  = prs[-1]               # called user name

    # Check CNAM first. If not found and US/German number, try to lookup caller's name in Whitepages or DasTelefonBuch

    if !(@cname = CNAM[@cid])
      if @cid =~ /^1([2-9]\d\d[2-9]\d{6})$/ && defined?(WP_key) # US/Canada pattern
        url = "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20xml%20where%20url%3D'http%3A%2F%2Fapi.whitepages.com%2Freverse_phone%2F1.0%2F%3Fphone%3D#{$1}%3Bapi_key%3D#{WP_key}'%20and%20itemPath%3D'wp.listings.listing'&format=json"
        if js = sys.WebGet(url,4).to_s
          @cname, dname, city, state = %w(businessname displayname city state).map {|x| js =~ /"#{x}":"([^"]+)"/; $1}
          @cname ||= dname; @cname ||= "#{city}, #{state}" if city && state
        end
      elsif @cid =~ /^49(\d+)$/ # Germany pattern
        url = "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fwww4.dastelefonbuch.de%2F%3Fcmd%3Dsearch%26kw%3D0#{$1}%22%20%20and%20xpath%3D%22%2F%2Ftable%5B%40class%3D'bg-01%20entry'%5D%2F%2Fdiv%5B%40class%3D'short'%5D%2Fa%22&format=json"
        @cname = $1 if sys.WebGet(url,5).to_s =~ /"title":"([^"]+)"/
      end
    end

    sys.Log("Caller's number: '#{@cid}'"); sys.Log("Caller's name:   '#{@cname}'") if @cname
    sys.SetFromHeader(formatNum(@cname || @cid,true), nil, Host)  # Set FromName & FromHost for sys.Dial
    sys.Dial("#{@user}@local[fu=#{@cid}]")

    else                    # Outbound call ...
I have checked this code with Bria and it works just fine (including umlauts, thanks to Aaron for prompt sys.WebGet fix). I didn't have a chance to check it with ATA or IP phone, will appreciate any input on this matter.

The code uses formatNum() method, don't forget to add:

require 'mikesgem'

in the beginning of your dialplan.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest