Thinking about how to go from directory listing to feed reader

I've been thinking about how best to track the technology blogs, the area in which I'll be serving as a contributing editor. I thought I might write a little program to grab the URLs of the technology blogs from the directory page, find the feeds, and construct some OPML that I could import into Bloglines. But then I looked at the directory listing and realized the links there are not to blogs themselves but to our descriptions of blogs. That adds a layer of difficulty to it.

I'm not thinking it's likely Drupal has some way of producing OPML that a newsreader could import, since that would involve hitting each blog site to try to find a feed, but I'm wondering if it can produce an HTML page with links to blogs instead of links to our descriptions of blogs... kind of a typical blogroll. If I had that, I have some Python code that will do the rest of the work. Not sure how long it would take me to get it working though.

I think there is a general problem of sync'ing this directory with the BlogHer site blogroll too. In the conference call, Lisa mentioned that the existing BlogHer site blogroll is maintained via BlogLines. Keeping that updated by hand sounds difficult and error-prone, especially as the directory here grows.

I'd be happy to brainstorm solutions to this and do what I can to help make it easier for contributing editors and BlogHer administrators keep up to date with the blogs in the directory. I don't imagine there are easy solutions at this point, but wanted to bring it up, because I'm feeling a little worried about the labor involved in following the blogs in the directory. And I'm not sure how to get all the current tech blogs into my feed reader except by hand.

Submitted by Anne Zelenka (view blog

) on Sun, 01/22/2006 - 14:05

Posted In

Comments (23)

Comments

Interesting notion

Pulling the feeds from the sites themselves into something people can use would be nice.

Our next step here is going to be to populate the aggregator (and in doing that decide which aggregator module to use here), and that's almost related.

Meanwhile, does this blogroll page help? (Links are redirected straight to the sites themselves.)

As for this site here, OPML info can be found here.

Submitted by Laura Scott on Sun, 01/22/2006 - 12:53

Thanks Laura

Laura - I ended up adding the feeds by hand, which was useful because I got a look at each one as I was adding it. Gets me oriented.

That blogroll page won't work for my purposes because I only want to track the tech & web blogs and it's just a big long list. The feeds on that other page are great--I'm now subscribed to the web & tech directory page so I should know if new blogs get added there.

Isn't it amazing how many different ways the same info can be packaged?

Submitted by Anne Zelenka on Sun, 01/22/2006 - 14:00

Hi Anne! I just added Diana

Hi Anne!
I just added Diana Eng to your blogroll because she seems to be a good fit. She blogs about fashion and her love for technology. Strange combo, but if you visit her link, and look on the right - her tagline really says it all.
Take care,
Karen
--
Troll Baby
http://bigdlittledmistatruffyandme.blogspot.com/

Troll Baby Graphics
http://trollbabygraphics.blogspot.com/

Submitted by Karen Rani on Mon, 01/23/2006 - 05:21

Thanks, Karen, for adding to the tech & web
blogroll

I will add Diana Eng to my own subscriptions so I can keep up with her blog.

Submitted by Anne Zelenka on Mon, 01/23/2006 - 08:56

There needs to be a way we

There needs to be a way we can automate the subscriptions and the changes, otherwise there is no way in keeping up.

Having a modul on this page may be nice for some people, but I want to read those feeds - old and new ones - in the feedreader of my choice. Having only an aggregator here is not an option.

Isn't there a way just to spit out the addresses with descriptions? It can't be that all is drupal centric and all it just going back to this page - I find that quite ego centric.

Also for blogher's following the according parts of the blogroll, I assume many would like to see the "newly added blogs" per category as well - and in this case also direct links and not something back to the drupal site. :)

Submitted by Nicole Simon on Thu, 01/26/2006 - 08:16

Short-term and long-term solutions

Hi Nicole - Yay! Great to see you.

And Anne, thanks for starting this thread.

Just want to confirm: You all are right - we need a solution. This is a top priority once we launch the site on Monday, Jan. 30. We are in the business of sending traffic to women bloggers. We are *also* in the business of helping users consume this content in any form they wish - as you say, in the feedreader of your choice, Nicole. So we will work on it and I know both Laura and I are eager for your help.

In the short-term, the easiest way to keep up with the site right now, IMHO, is the "recent activity" under the Shortcuts menu. It shows you the latest posts and comments since you last logged in. I realize this is site-based! But we are indeed trying to serve you in every way we can...and still launch.

Best
~L

Submitted by Lisa Stone on Thu, 01/26/2006 - 11:02

If that's the case can we provide full feeds

"We are *also* in the business of helping users consume this content in any form they wish"

If that's the case, can we provide full feeds? I've subscribed in bloglines and I'm only getting a few sentences.

I've already lost (?) the "battle of the comments formatting" and the "battle of the blog text on the front page", so I'm hoping you'll give me this one... ?

P.S. - Can't believe nobody's commented yet - love the new picture, Lisa. Very artistic!

Melinda

Submitted by Melinda Casino on Thu, 01/26/2006 - 11:30

I cannot speak for Bloglines

But feed content length is controlled by the source blog's settings. For example, our feeds coming from here will show only the teaser, with a link to the full post. Many blog softwares are set to show only the first x words, and many bloggers don't want to offer full feeds lest people never go to their sites.

I assume you know this, but I'm posting this for the benefit of others who might wonder about this.

Submitted by Laura Scott on Thu, 01/26/2006 - 11:44

Thanks, Laura

Thanks, Laura, but I was wondering if there were some way, through the magic of RSS, to provide full feeds to people subscribing to the BlogHer site. I believe this is possible even when "read more" stuff is used by bloggers.

I don't think providing full feeds will really hinder how many hits this site receives, because if people enjoy a piece they usually want to read the comments... and also join in.

Speaking of comments - is this issue decided?
Either Laura or Lisa, please fill me in here, your silence is deafening. (For other readers, see my post in the Forum section.)

Melinda

Submitted by Melinda Casino on Thu, 01/26/2006 - 11:52

Thanks Mel! My nine-year-old

Thanks Mel! My nine-year-old took the photo. I abhor cameras for many reasons, but I do find this particular photographer irresistible...

Laura makes a great point re: site visits. And yet I'm happy to consider issuing full feeds from BlogHer. In fact, my goal would be to provide the following choices:

- Full feed
- Headline and excerpt (way I prefer to read them)
- Headline only (for those folks who are browsing via PDA

I'll put it on HRH Laura's loong list for us to discuss. Thanks mucho!

Best
~L
Lisa Stone

Submitted by Lisa Stone on Thu, 01/26/2006 - 11:49

We must've written our comments at the same
time

Hi Lisa - Aw, that's cute! The picture is even better knowing your nine year old took it...

I must've been writing my comment when you were writing yours...

Full feeds: WHOOOOOOOOOO - HOOOOOOOOOO!!! *jumping up and down excitedly* I realise it's only on the "to do" list, but at least it's there. Thanks - I'm sure many others will appreciate this as well. Not just vocal tossers like myself. :D ;)

Melinda

Submitted by Melinda Casino on Thu, 01/26/2006 - 11:55

That will take hacking

I will look into it, but this is not how the Drupal core code functions. And going through something like Feedburner won't work, either, since all they would get is the teaser, too.

Please consider this a Beta site issue, with no guarantees on results. I suspect it's not too terribly involved, but one never knows.

Submitted by Laura Scott on Thu, 01/26/2006 - 12:15

Duly noted!!

Noooo more hacking. Until launch...

Best
~L

Lisa Stone

Submitted by Lisa Stone on Thu, 01/26/2006 - 13:08

Noted

OK.

*not jumping as excitedly now*

Melinda

Submitted by Melinda Casino on Thu, 01/26/2006 - 17:11

I hope I do misunderstand

I hope I do misunderstand you when you say that the drupal core does not support full feeds but just teasers?? That would be broken by design.

Submitted by Nicole Simon on Sat, 01/28/2006 - 09:58

Drupal feeds

Drupal feeds are derived from the teaser display. The feed is as long as the teaser. If the teaser is full-length, then the feed is. If that's "broken by design," I suppose that's because nobody yet has patched the code to make it do otherwise.

Submitted by Laura Scott on Sat, 01/28/2006 - 19:00

FYI BlogHer's RSS feeds are now full-length

It's a rather ugly hack so please let me know if there are problems.

Note: ALL posts are now full-length in the RSS feeds.

Submitted by Laura Scott on Sun, 01/29/2006 - 13:14

RSS hack rocks!

Thank you Laura! Look at that: I'm moved to exclamation points.

Lisa Stone
BlogHer Co-founder | Surfette

Submitted by Lisa Stone on Sun, 01/29/2006 - 13:24

Agreed...

It looks terrific. Holding the blockquote and adding the permalink in there, too. Terrific!

~Denise

Submitted by Denise on Sun, 01/29/2006 - 13:26

You can subscribe to blogher blogroll changes

Nicole, check out the syndication page. It has links to feeds of all sorts of useful things, including the blogrolls themselves. I'm subscribed to the technology blogroll feed so that I see when new blogs have been added. Then I can add them to my subscriptions.

Submitted by Anne Zelenka on Thu, 01/26/2006 - 11:16

Hi Anne, yes I saw that, but

Hi Anne, yes I saw that, but that is only a second best option for me, as this all - again - goes through the blogher site. :(

Submitted by Nicole Simon on Sat, 01/28/2006 - 09:59

Nicole, you can get links that take you
directly to blogs

Nicole, I may be missing something, but I think you can get pretty close to what you want. I am subscribed to both the technology & web blog feed and the technology & web blogroll feed. The latter feed provides an RSS article for each new blog added to the tech & web blogroll. The body of the article is just a link to that blog, not through the BlogHer site. I click through to the blog, check out if it looks like one I want to follow, and add it to my BlogHer - Tech & Web subscriptions if so.

True, it's not entirely automated. You have to go to the blog and subscribe yourself. Ideally we need a reading list, i.e. regularly updated OPML listing feeds living at a URL to which we subscribe. That would require reading list aware feed readers, like BlogBridge. And it would require BlogHer to find the feeds for each source.

I'm working with OPML and reading lists right now to see how I can automate it better for myself and eventually for others. For now, I think subscribing to the blogroll itself is okay.

What do you think? Does this meet your needs at all or am I just confused (happens quite often :-)

Anne Zelenka
BlogHer Contributing Editor, Technology & Web

Submitted by Anne Zelenka on Sat, 01/28/2006 - 11:05

More on OPML

We will be able to offer up an OPML file of all of our aggregated content from elsewhere. We just have not populated the aggregator yet, so there's not much to import. That will happen soon, though.

Submitted by Laura Scott on Sat, 01/28/2006 - 18:56