Getting Ad Insights from Facebook – and avoiding error 17

At untapt measuring the success of our marketing campaigns across social media is important – to us and our clients. If we are going to put dollars into a campaign to find the best tech talent around the world, we need to know which social media platforms are performing best.

I’ve struggled with the Facebook API so I thought I would do a quick recap of the solution I implemented in Python. In doing so, I had to make sure that the number of API calls was limited so avoid Error 17 – user request limit reached.

The most important discovery in the code below was that insights can be retrieved for the account, but to see them aggregated at the ad level you simply need to pass the ‘level’ parameter.

Shout out to http://jakzaprogramowac.pl/pytanie/12122,fb-ads-api-17-user-request-limit-reached for pointing me in the right direction and also to this Stackoverflow response which was essential in helping me converting the access token I retrieved from the Facebook Graph Explorer to a long lived access token.

Node.JS and Heroku

I am following Node:Up and Running to learn about Node.JS and extending what I have gleaned from this book, and learned on the excellent Javascript courses at O’Reilly School of Technology, so that I can deploy the book example apps on a Heroku dyno.  I like the way the book is written, but there are some really frustrating gaps – thus far.  At least for a node.js n00b like me.

If you got stuck like me, here are some fixes.  Most of these stem from a) typos and b) I assume, earlier versions of EJS and Express.

Chapter 2 – Example 2-1. A test for the POST API

The twitter_test.js file fails.  This might be by design – but if you want to correct the test case then you need to fix the assert.strictEqual to equal res.send.  The complete file then becomes:

Chapter 2 – Examples 2-17 through 2-22.  Getting the Chirpie site to work.

First off, you need to install EJS.  Do so from where your project is – so in my case, run it in the ‘my app folder’ – see below.

Run:

In the node_modules folder you will see a folder for ejs.

What the book does not state is that the example code needs to be saved in files with a .ejs extension.  So index.ejs should exist in the views folder and chirp.ejs in the partials.  In addition, the body tag is no longer support so you need to break the layout of the page into the ‘top’ and the bottom’.

The folder structure, with files, now looks like this:

The content of the files is as follows:

app.js

index.ejs

layoutTop.ejs

layoutBottom.ejs

chirp.ejs

stylesheet.ejs

Now when you run you should see the the Chirpe page with an input box.  Enter some text and hit ‘tweet’.  You should see each tweet (aka ‘chirp’) appear below as a comma-delimited string.

 

Lego Digital Designer

The Lego Digital Designer is an awesome tool for designing a new creation on your computer and creating a build guide. You can then use the Lego Pick a Brick site to buy all the bricks you need.  A Raspberry Pi case can be put together for ~$20.

I’ve attached my own design which you can download – I used Biz’s excellent instructions as a template (thank you!).

There are some caveats however:

  • You can’t automatically generate a shopping list from the LDD – you need pick the bricks manually, which can be a tedious exercise.
  • To rub salt into the wound not all the bricks in LDD are available on the Pick a Brick site – so you need to switch back and forth between LDD and the Pick a Brick site to double check you are using bricks that you can actually purchase.

Raspberry Pi useful links and notes

Links

Notes

  • When creating the image using Win32 Disk Image, initially Win32 Disk Image kept crashing.  The .img file was on my D:\ drive.  Moving it to somewhere on C:\ (e.g. C:\temp) resolved the problem.
  • If the Pi keeps rebooting it is probably due to low power voltage.
  • I had black lines across my screen initially.  Turns out this was due to a faulty HDMI cable.
  • You can run Flash if you install Gnash.  However it is not up to date enough to run Scratch in the Browser.  The Scratch app that comes with Raspbian is cool but appears to lack some of the features – which is a shame as I was hoping my 6 year old who has been learning Scratch would be able to continue on the Raspberry Pi.  But to be honest I have not looked very deeply at this yet.

Setting up Heroku, Flask and Python on a Windows 7 PC

Following the steps on the Heroku site for getting started with python guide I ran into a few issue-ettes.  Below are some scrappy notes that could be used to accompany the guide if you run into the same issues that I did.

Init()

Before using the quickstart guide on the Heroku site itself, you’ll need to have ssh-keygen installed.  The Open-SSH package for Windows can be downloaded from here: http://sshwindows.sourceforge.net/download.  On installation you don’t need the server component and the PATH variable should be updated automatically to include the location of \bin\ssh-keygen.exe.  WARNING: My PATH was destroyed twice after installing SSH.  I would back it up first!

Once the Heroku toolbelt is installed and you login for the first time, when prompted to generate an SSH public key, the process will find and run ssh-keygen.exe.

Can’t find Foreman

I followed the steps here suggested by Mark Rajcok: http://stackoverflow.com/questions/19078939/foreman-installed-by-heroku-toolbelt-on-windows-cant-be-found successfully to get Foreman working.  Note also below the comment on running without gunicorn.

The main points for me were:

  • install Heroku to c:\heroku (nowhere else)
  • add C:\Heroku\ruby-1.9.2\bin to the PATH
  • I did not need to un/install Foreman or downgrade it from version 0.63

Sample Python code gives a 404 error

The hello world Python code here lacks code to run the app.  This is not the complete solution with respect to a Heroku deployment – see further below.

Foreman found, but python code still does not run.

Don’t use gunicorn, so the contents of the Procfile should therefore be:

Push rejected, no Cedar-supported app, detected.

Did you create the requirements.txt file?

 Heroku deployed but app crashes with: “Web process failed to bind to $PORT”

Heroku assigns the PORT, so the Flask default PORT of 5000 cannot be used.  The python code should therefore be changed thus:

 No web processes running

I did not realise that I need to start up the web service manually after each deployment.  Running heroku ps:scale web=1 resolved the issue.  You can then also run

to verify the web process is up.

Resynching a repository or setting up a new machine

re-synch with:

For a new machine generate an ssh key and add to Heroku.  See https://devcenter.heroku.com/articles/keys. When I did this initially at the office I had lots of problems with permissions – i.e:

The problem was that the key was being generated on my HOMEDRIVE (in my work this is F:\.ssh) but Heroku was looking in C:\Users\MYNAME\.ssh.  Copying the id_rsa files from F:\.ssh to C:\Users\MYNAME\.ssh fixed the problem.

 

HTML, CSS and javascript Useful Links

vCheck – VMWare Daily Report

vCheck is a series of scripts (or plug-ins) written by Alan Renouf that can be used to report on the status of your VMWare environment. It can be run from VMWare’s Powershell.

Note that these scripts are not an audit or a full report but list potential issues or warnings. Further, many/some of these might not be relevant in your particular environment.

See the following links for more information:

I found the plugins particularly useful to report on all snapshots across the VM environments Ambre manages.  There are over 70 plugins including, for example:

  • Hosts not responding or Disconnected
  • Created or cloned VMs
  • Removed VMs
  • VMs with over CPU Count
  • VMs Swapping or Ballooning
  • Invalid or inaccessible VMs
  • No VM Tools
  • Hosts with reboot required
  • Powered Off VMs
  • Snapshot created

WrightCCS is a viable alternative to RSA for dual authentication

Now that RSA cannot even be trusted, it might be time to look at alternative dual authentication options.  One that we have had success with at C12 is WrightCSS.  WrightCSS is a member of the Citrix Partner program and its SMS2 offering has been designated as Citrix Ready.

The WrightCSS Admin GUI sets private PINs and allows one to select from a choice of having passcodes delivered via SMS text messagingEmail or OATHCalc.

We have not tested WrightCSS with physical tokens.  If the OATHCalc option is enabled the user can download and install a soft token application such as Google Authenticator to a smartphone.  This product can be installed from the apps store for an Android or iOS device.  For BlackBerry smartphone, users can download the app from http://m.google.com/authenticator.  Note, the BlackBerry version is a bit ‘rustic’ and we’ve found it to not refresh as well as the Android and iOS versions.

When choosing the OATHCalc option, a user can configure the Google Authenticator by manually inputting the username and the Shared Secret (automatically generated by SMS2).  Alternatively, after hitting Save Configuration, a QR code will appear.  The QR code can be saved to the clipboard and emailed to the user, or if the user is at the admin’s desk (or self-service is enabled) the user can hold the smartphone up to the screen and capture the QR code.  Doing so will generate and insert the token into the Google Authenticator app automatically.

auth_options

To date, we’ve found SMS2 to be a robust, cost-effective (licensing is FREE) dual-factor authentication solution.  This isn’t specific to Citrix either as it can work in conjunction with other applications that support RADIUS.  

Use Microsoft Log Parser for trolling through the Event Viewer

Sifting through the thousands of entries in a server’s local Security Event log for a specific message can be a very time consuming experience.

One way to quickly sort through the noise is to use Microsoft’s Log Parser (FREE!).  Log Parser is a command line utility that uses a language structure very similar to SQL.  Install it on your PC and manually add C:\Program Files (x86)\Log Parser 2.2 to your PATH.  (Right-click My Computer | Properties | Advanced system settings | Advanced | Environment Variables | System variables | Path | Edit).

Log Parser Syntax

As stated previously, Log Parser commands are very similar to SQL queries.  Let’s say you are looking for an event where the EventID = 4656. You could type the following command:

If you are running this on the server, you can simply state the source to be “SECURITY” as shown above. Windows will know you are referring to the server’s local Security log.  If you are running this on your own PC, you would type:

The above -i:evt switch means the file being used for input is an EVT file (which is the file type for Event Viewer). Alternatively, if you were searching against a text file for some other usage, you could simply put -i:txt.  When you run the above commands, Windows will attempt to search the entire file.  However, Windows will only output 10 entries at a time and you will get the “Press any key to continue prompt…”  If you wanted this to search the entire file at once, you can add the -rtp:-1 to the end of the command.

For example:

Unfortunately, you will see Windows whiz by potentially hundreds of entries on your command prompt.  The solution is to pipe the results to a text file.

For example:

Unfortunately, we’ve added too much information into the text file. Even with Notepad++ or TextPad, reading through all this data can be time consuming as well.  What would help is if we knew all the field types within the Event Viewer so that we could narrow things down a bit. Below is a list of fields used within the Event Viewer:

We can now be a bit more creative. Let’s do a search for the same EventID but limit the output to display only the TimeGenerated.  Moreover, if we suspected that David Im altered the file, let’s modify the search to only include where his username appears (ex: sam). We could type:

What if I want to automate this as a task? Well you can put this into a batch file and run it as a task via native Windows Task Scheduler or VisualCron.  However, some modifications have to take place.  For example, the % is a special character within Windows batch files that signals to the system that you are about to refer to a variable.  Thus, for Message LIKE %sam%, you must modify the batch file to be Message LIKE ‘%%sam%%’.  Notice it also required the single tick marks.  You would also need single tick marks around EventID = ‘4656’.

Unfortunately, the above could still display (potentially) hundreds of entries. What if we want to narrow it down to events that occurred in the last 5 minutes? There is support for that as well.  The syntax below assumes you are putting this into a batch file:

 

Sending SMS messages using Twilio

Twilio provides an API and platform that can send SMS texts, and text-to-voice messages, and costs very little to use.

Some 3rd party vendors already have hooks into Twilio – e.g. WrightCCS for two-factor authentication.  If you want to roll your own, however, then creating an app in C# is very simple using the examples here.

Such an app can now be called in a variety of ways.  I’ve personally implemented it in the Frameflow monitoring tool and VisualCron – an excellent Windows task scheduler.  These were of particular importance for managing alerts when the application impacted is Microsoft Exchange (because obviously no emails alerting IT Support could be sent).