Extract status-posts and comments from public Facebook pages

Facebook loader is a simple web application, where you can get status-posts from public Facebook pages in a given time span. Its also possible to render the result in a tab-separated format and then copy/paste into Excel, Word or another editing system for further analysis.

Important legal notes

Information can not be extracted via facebook loader for research or other purposes, without having any legal basis. This means either consent from each person you retrieve data from or pursuant to law. Researchers are responsible for obtaining this before using the application. In addition, there are clear rules on what applies in order to authorize a research project to investigate the personal data. University has appointed NSD as the institution's privacy representative in research. If questions or in doubt contact datafangst@usit.uio.no or read more on the UiO personal privacy page (in Norwegian).

Log in to Facebook first

The application works only properly if you are logged in to Facebook with the browser you are running the application with .

If you are not logged in, a Facebook-window should pop up and ask you to log in. If your browser is set up to prevent pop-up windows, this window will not be shown and the application won’t work. It is then only possible to get metadata but not posts. If this is the case, go to facebook.com and log in.

Find username/ID to the public page

The next thing is to find out is the specific/ID for the page you are interested in getting posts from. Example:

If you searched for the name Jens Stoltenberg in Facebook and found his public page, the URL will be: https://www.facebook.com/jensstoltenberg

 

 

The username/ID you should use in facebook loader is then: ‘jensstoltenberg’

However, sometimes the URL will look a little different – here is an example with the Danish politician Uffe Elbæk:

https://www.facebook.com/pages/Uffe-Elbæk/235501083187697

The ID in this case is a serial number (235501083187697), copy and paste the number into the username field. When the username is found click the button ‘Get metadata’

Get the posts

Start facebook loader

Before you click ‘Get Posts’ you must at least choose a start date so the application knows how far back it should go. The end date is optional and if you don’t specify anything the end date is today. After setting up a date, click the button ‘Get posts’.

In the gray field ‘Now loading posts’ you can see the application is working. A date plus the number of posts until now is shown in the field. The application will always start with to day and move backwards, but if you have specified an end date the program will later on throw the posts created after that date away. When all posts are fetched a window will show up telling that the job is done.

Explanation of the various fields

The following fields from each post is written out in the text-field

  • PAGE: The name of the page
  • TITLE: The title on the post. If a post does not has a title start of the body text is copied into the title
  • FROM: The name on the person that has written the post
  • LINK: An external link if there is one
  • CREATED: The creation date
  • LIKES: Number of likes
  • SHARES: Number of shares
  • COMMENTS: Number of comments
  • SEE_STATUS: The url to the post
  • MESSAGE: The body text

See a post

Now you will see a list box with the titles to the uploaded posts and the fields to the individual posts, by clicking in the list box on the post you are interested in. You can read the text in the post and get a link to the post in facebook, and see how many likes/shares and comments the post has. One additional button is created. If you click on the 'Comments' button all comments to the post will be exported to a tab-separated text field.

Generate a tab-separated format for export

If you want to write all the posts out in a tab-separated format you can click on the ‘Start’ button. In the grey field ‘Loading’ you can see if the application is working. This process can take a while depending on how many posts there are. If you discover, that the program has stopped – the content is not changing in the Loading field anymore -  do the following:

Copy the result you have got so far into Excel or another editing system, and there you can see how far the process has come. Then you make a new upload from that time and run the proces again and merge the results. It happens that facebook throws you out of the process.

Collecting photo information from Facebook

Metadata in photos are handled in a slight puzzling way in Facebook. If you extract metadata from posts registered as photos the number of 'likes' to the post is in fact the number of 'likes' to all the photos in the album.

We have therefore made an application only for photos, which give you the right numbers of 'likes' and where you also can see the photos and the comments to a given photo.

Showing photos

When you have typed in the Username/ID and clicked on the button 'Get metadata', the cover photo of the page will show up and subsequently a list box with all the titles of the photo albums for that public page (or user if you are fb-friend with the person concerned).

If you double-click on an album title in the list box, a table with thumbnails of all photos in that album will be created. Additionally, if you double click on a photo you will see the photo in full size. The number under the thumbnail is the actual number of likes, if you click on this you will be taken to the specific Facebook post where the photo is used.

Exporting metadata

The album list box is a multi selection list box. When you click 'Export photos' only the selected albums will be exported.  You may split the export in as many portions as there are albums. The export makes a lot of calls to Facebook, and sometimes Facebook will throw you out and stop answering requests for half an hour. To avoid this, we recommend splitting the export if it is a page with many photos.

When the export is finish a text field with tab-separated text will appear and you can copy and paste the text in wherever you like (e.g. into an Excel spreadsheet).

The fields exported are:

  • ID: the uniqe serial number for the photo
  • THUMBNAIL: a URL to a thumbnail of the photo
  • SOURCE: a URL to the photo in full size
  • LINK: a URL to a Facebook post where the photo is used
  • CREATED: the date where the photo is uploaded in Facebook
  • LIKES: number of likes
  • COMMENTS: number of comments in all

Individual objects

If you want to investigate an individual object, you can instead paste the object id to the field 'Single object id' and click on 'Load single object'. The object can e.g. be a status, video, photo or a link. The object will then come up and you can export comments to the object in a tab-separated format. This feature functions only for objects in pages, since Facebook has restrictions on harvesting user data even when the data are public.

Published Oct. 25, 2013 9:38 AM - Last modified Mar. 20, 2017 4:13 PM