L&LR staff blog

Joss Winn from the University’s Centre for Educational Research & Development has already explained how to generate some code which creates a dynamic list of an author’s publications, drawing on an RSS feed of items in our Repository, for embedding in another webpage (e.g.  a staff profile page on the University’s website).

(Upfront, I’ll say this – I agree that it would be much better if we could do this with one click of an ‘embed my list‘ button. Until that day arrives…)

But: there’s a problem with the RSS feeds that come out of the EPrints software:

The items in the list don’t appear in any particularly logical order, and the order can’t be changed. (In fact, they’re displayed in reverse order by the date they were added to the Repository, which has its own internal logic, but makes no sense ‘in the wild’.)

To compound this problem, the actual date of publication is buried rather deeply within the item description, e.g.:

Neary, Mike and Winn, Joss (2009) The student as producer: reinventing the student experience in higher education. In: The future of higher education: policy, pedagogy and the student experience. Continuum, London, pp. 192-210. ISBN 1847064728 (In Press)

The challenge is to extract this date of publication and use it to re-order the items in the list.

I’ve used Yahoo! Pipes to do this, by:

1. Copying the description into a new field.
2. Using the following Regex to isolate and extract the date of publication: ^.*\((\d{4})\).*$
3. Re-ordering the field by this date (in reverse order).

You can find the Pipe at: http://pipes.yahoo.com/lincoln/eprints_sort2

(Image: screenshot of the Yahoo! Pipe)

The final enhancement – I added user-input fields to allow you to use the Pipe to create a feed for any author (forename/surname), which can be further refined by adding an extra keyword (we’re using additional keywords, added to the EPrints record, as a secondary hack to distinguish between multiple authors with the same name!).

Once the Pipe has run, you can grab the RSS feed and pass it through Feed2JS in the usual way.

This isn’t particularly satisfactory in the long run, in that it relies on a lot of external services over which we have no control, but it’s a useful hack for now.

Hopefully, you should have noticed a few recent changes to the Electronic Journals A-to-Z.

• First, we’ve all but completed the summer campaign to improve the accuracy of the 60-odd individual e-journal package holding files. Di, Adele, Carole, Phil, Elif and I have been working through each package in turn and using the most up-to-date information from the publisher/provider to improve the A-to-Z holdings. One or two inconsistencies still to iron out, and it’ll never be entirely bug-free (please carry on sending any mistakes that you spot to acquisitions@lincoln.ac.uk), but the quality of the data for our managed packages should be significantly better than it’s ever been. This review process is going to be [it's going to have to be] an annual event.
• Second, the ‘New e-journals at Lincoln‘ RSS feed has been updated through the summer to reflect the changes. Noting that EBSCO’s terms and conditions for use of the A-to-Z state that we ”may not distribute, encourage or allow distribution of [...] data updates“, I’m going to be restricting the RSS feed to just those packages where we, not EBSCO, are the source of the updated information (i.e. where we are managing the title lists directly). This includes the packages providing access to our individual title-level subscriptions: i.e. the ‘important stuff’.
• Third, and last of all, as I discussed with several people in my workshops at the L&LR awayday in June, I’ve revamped the A-to-Z home page ready for the start of the 09/10 session…
  As discussed in those workshops, I’ve taken a ‘just do it!’ approach. It’s not the finished article by any means – I’m relying on your initial and ongoing criticism, comments and ideas to take it forward – but we agreed: better to make changes then tweak as appropriate, rather than discuss ouselves into inaction.

(Image: the new A-to-Z homepage)

Compare with the old home page (below):

(Image: the old A-to-Z homepage)

The changes / new features include (hopefully these will ring bells with those of you who took part in the discussions):

• Generally – the aim has been to make the site more clean and simple.

1. The page ‘header’ – extraneous information (and slightly excitable formatting!) removed – just the name of the application (there was some discussion about dropping ’Electronic Journals A-to-Z’ in favour of ’E-journals Catalogue’ or similar, but now’s probably not the time for that) and the strapline with the number of titles.
2. The navigation tabs – I had hoped (and people had agreed) to reduce these to a minimum, replacing most of them (advanced search, subjects, etc.) with lower-profile links further down the page, but I encountered a problem – if I use A-to-Z admin to ‘hide’ a tab, it becomes impossible to link to that tab elsewhere. This is weird, and unfortunate in the extreme – unfortunately it’s something we’re stuck with for the time being. Inevitably, I’ve ended up adding to the number of tabs – as well as a new tab for the new home page, there’s also now a ‘Log in‘ tab which routes the user through Athens (and back to the homepage), which ought to help us to troubleshoot users’ full-text access problems.
3. A big, unmissable search box. Needs a better search button graphic, but it does at least now include a search tip (to subtly reinforce the idea that users should be searching for journal titles here).
4. See also the sentence on using the e-Library for initial subject keyword searching  – could we / should we expand on this?
5. More options – links to the other A-to-Z pages. I had intended these to replace the tabs entirely, but see (2). Maybe, given that, we don’t actually need these extra links?
6. Still to come – a screencast video showing the user how to use the A-to-Z in the context of a literature search. I’ll blog when that’s ready. Also need to revamp the ‘Help’ tab more generally.

So – what do people think?

I’ve shown a couple of people this idea that Julie and I have been developing for Holbeach – that of using RefWorks to manage a new-books list for the relatively small number of new titles that we receive each year at the campus LRC – and that can be embedded into Blackboard or a subject web page.

The advantage of using RefWorks (apart from the fact that many L&LR staff are reasonably familiar with it) is that the RefWorks shared folders do most of the work for you.

If people like it, there’s no reason you can’t start using it straight away. I’ve already a trip planned to Hull next month to see if it’ll work for colleagues there.

First, here’s the end result: http://feeds.feedburner.com/HolbeachLRC

Now, here’s how it’s done:

• Every time Julie gets a PO slip back for an order she placed, she logs into her RefWorks account, searches our catalogue for the book, and imports it.
• Julie has created a RefWorks folder called ‘Holbeach new books‘, and shared it publicly. All newly-arrived titles are added to this folder on import.
• This is the clever bit – RefWorks allows you [PDF, p.4] to make the contents of a shared folder available as an RSS feed. Here’s the feed for the Holbeach new books folder. You can treat this like any other RSS feed – follow it in Google Reader / wherever, embed it into a web page using Feed2JS, etc.
• But… it’s not very pretty.
• So, in comes Yahoo! Pipes. I’ve created a pipe which takes any RefWorks shared-folder RSS feed with books in it (not designed to work with journal articles or other items – perhaps that’s phase II). This pipe, which you can find at – http://pipes.yahoo.com/lincoln/newbooks - does the following:
  1. Looks for, and extracts a valid(ish) ISBN from each RefWorks item;
  2. Creates a new link to our catalogue which looks up this ISBN for each item;
  3. Displays a book-cover image from Amazon.co.uk matching that ISBN (with a link back to Amazon through our affiliate scheme), next to a description of the book (manually added to RefWorks), and an extra link back to the original item record in RefWorks (marked by the RefWorks icon: ) – I’ve just added that last element, today, because I think it’ll be useful to give students & staff the option to take each item and export it to their own RefWorks account. Also to give credit where credit’s due, and recognise that RefWorks is the source of the information!
• The ‘prettified’ RSS feed from Yahoo! Pipes is then fed through Google Feedburner, in order to give it a stable, sensible URL, collect stats on subscriptions and hits, and to allow email subscriptions.

Couple of slight problems / areas for development: we’re finding it necessary sometimes to ‘tweak’ the ISBN before it can find an Amazon cover image. Maybe with a better use of regex in my pipe, that wouldn’t be necessary? Also, at the moment Julie is manually adding the paragraph description to each item after import. I’m sure that somewhere out there exists a source of book descriptions / reviews that I could pull in automatically (hello, library mashup fans?!) – that would streamline the process a bit.

Edit (8th July) – slideshare.net does not seem to like displaying these slides on-screen, so here they are to download in MS PowerPoint format.

Here are the slides (on slideshare.net) of the 5-minute lightning talk I just gave at Mashed Library 2009, on using ticTOCs data to display e-journal ToC feeds, and on creating a new-titles feed with Yahoo Pipes, etc.

…whether the subject is food or finance, or – in theory – any keyword you want, you can filter the RSS feed of new e-journals by constructing a link that looks like this:

• http://pipes.yahoo.com/pipes/pipe.run?_id=mDjFxR1R3hGHC2SH0z6skA&Subject=XXXXXXXXX

Where “XXXXXXXXX” is your subject word.

That link creates html which you could link to within Blackboard (see illustration below – click for bigger). Or, to get the titles as RSS, just add &_render=rss to the end of the URL.

This is done using Yahoo! Pipes.

You can now subscribe to my RSS feed of new e-journal titles via email, should you so wish.

• Here’s a link to the email sign-up form.

I prefer to keep up-to-date with RSS feeds using Google Reader (I don’t want or need anything more in my Outlook inbox!), but if email’s your bag and/or you prefer to have all your information in one place, then at least there’s now the option to add these to your inbox.

I have a feeling that there’s just too much in this feed (c.90 new titles added over the Bank Holiday weekend) to be useful as a public i.e. library-user-focused service. Too many titles, no subject filtering (yet…), too much (for want of a better word) noise in the form of low-academic-interest titles added to packages such as Factiva.

But I do think this could be a really useful current-awareness tool for Academic Subject Librarians.

In February, I wrote about  how I’d added more than 5,000 ticTOCs RSS feeds to the A-to-Z, so that researchers can easily find and subscribe to Table-of-Contents (ToC) updates from journals in their field. I think this is an indispensable current awareness service, and I’m pleased it’s being promoted at the ‘Working Smarter With the Web‘ workshops.

(An aside: that ticTOCs data is overdue to be re-generated and reloaded onto the A-to-Z. This is next on the long list of e-journal packages for the “e team” to review.)

That’s all well and good, but there’s a clear need for a current awareness service ‘one level above’ the individual ToCs. That is, we could (and should) be providing an RSS feed of new e-journal titles.

At the moment, the EBSCO A-to-Z platform doesn’t provide this feature (I really do hope it’s something that EBSCO are considering developing themselves), and so in the past we’ve had to resort to emails, spreadsheets, etc.—all with varying degrees of success—just to keep people up to date with what new e-journals have become available.

~~~

So, I’ve created a rough-and-ready working demo of a ‘New e-journals at Lincoln‘ RSS feed for the A-to-Z. It’s not supported by EBSCO, so it’ll require a bit of manual intervention to keep it running (it’ll be a nice five-minute daily task for the “e team” to take on!), but has the potential to be a really important enhancement to the e-journals service.

Next time you go to the A-to-Z site, you’ll see a new tab – next to the existing ‘Titles’ tab – which I’ve labelled ‘New Titles (RSS)‘. This new tab links to a page containing information about the RSS feed, with a prominent link to the feed itself, plus a display of the 10 most-recently-added titles in the feed.

The links in the ‘ten most recent…’ list should go directly to the A-to-Z record for that title. The feed link itself you can paste into your favourite feed reader (I use Google Reader). I’m working on enabling email subscription to the feed. We should also be able to take this feed and filter/manipulate it to create (for example) subject-specific lists in Blackboard.

I’ve also made the feed autodiscoverable (c.f. my earlier post), so it should display in the browser toolbar (this will vary between browsers).

This is how the feed is auto-discovered by Firefox browser.

I still need to do some work on creating better documentation for users of the feed, and for the process of adding new titles within L&LR.

I’ll also need to do a bit of testing to make sure everything’s working as it should be, and iron out my feed-validation bugs.

When the time is right, I’ll launch the service more publicly.

Paul

~~~

P.S. Here’s how I did it:

1. The first stage is the most clunky – it involves getting information about the newly-added titles out of the closed box that is the Electronic Journals A-to-Z. EBSCO provide a daily report (available only to administrators) of “Titles added to Packages in my Collection“. This data can be exported on a daily basis in tab-delimited format.
   
2. Next, I created a spreadsheet using Google Docs and published it to the Web. The tabbed data from the A-to-Z admin report is pasted in to the spreadsheet, so it becomes openly accessible. Using Google Docs also means that I can invite other people to act as editors of the spreadsheet, and create public web forms to add titles that don’t appear in EBSCO’s daily report (not all of our e-journal packages are managed by EBSCO, so the data will need supplementing when we add custom titles).
3. The data output of the Google spreadsheet is then fed into Yahoo! Pipes, an excellent application for ‘mashing up’ and processing data. You can inspect the Yahoo! pipe for yourself, at: http://pipes.yahoo.com/lincoln/newejournals. If you create an account on Yahoo!, you can clone this (or any) pipe and modify it to create your own data mashup.
   
4. The Yahoo! pipe takes the raw Google spreadsheet data, filters it according to various rules, and exports a valid * RSS feed of new items, in reverse order by date.
5. * Well, it’s nearly valid. I need to do a bit of cleaning up of the pipe (I did say this was a working demo), mainly so I can pass the feed through Feedburner, which will allow email subscriptions. But it works well enough!
6. Finally, I created the new tab on the A-to-Z public site, and added the details to the page (the ‘ten most recent’ list was created using feed2js). The code for the autodiscoverable feed was pasted into the top of our customised A-to-Z template.

~~~

For anyone who doesn’t use Yammer and who missed the conversation leading up to this: data innovator Tony Hirst of the OU posted to his blog explaining how he’d…

“put together a page showing how the well the HEI Libraries are doing at getting autodiscoverable RSS feeds declared on their website homepages.“

Only about 10% of the library homepages had autodiscoverable feeds, and Lincoln wasn’t one of them. A conversation with Joss and the University’s website manager later, and now our feed is there:

A tiny thing, and who knows whether it’ll lead to greater readership of the news blog in future? But worth the small investment in time, I feel.

I’m at Birkbeck today for Mashed Library ‘08. Here’s a feed of my notes on a Learning Lab (micro)blog set up for the occasion.

[cetsEmbedRSS id='http://learninglab.lincoln.ac.uk/blogs/mashlib08/feed/' itemcount='0' itemcontent='0']

Paul

P.S. I am going to write this up formally, if you can’t follow my stream of consciousness notes…

A few people have asked if we can (and whether we should) set up individual subject-specific blogs.

The thing is, we’ve already got ‘em (kind of):

• Architecture
• Art & Design
• Forensic Science
• Humanities
• Journalism
• Law
• Performing Arts
• Policy Studies
• Psychology
• Social Work

Each link is a separate RSS feed of postings to the main L&LR news blog, filtered down to only those posts which have been tagged with the name of a subject. Any new posts you add with the same subject tag will show up in the feed. And you can create a new subject feed by writing a post and creating a new subject tag.

These feeds could then by displayed in (e.g.) Blackboard using a tool like feed2js.org, which renders the RSS as JavaScript code (which you can then paste into an item in a Bb folder).

And if we ever wanted to have entirely separate subject blogs in the future, we could just convert the tag-specific feed into a separate named blog. Semantically there’s no difference in WordPress between a blog, a category, and a tag.

Paul