« OpenFLの未読最大件数を200件にした | ホーム | Plaggerをcronで定期実行する »
2008年5月26日
OpenFL + Store::Fastladderで広告エントリーの削除と全文取得をする
せっかくPlagger通してるんだからやってみた。
yamlはこんな感じ。
plugins:
- module: Subscription::LivedoorReader
config:
username: USERNAME
password: PASSWORD
- module: Filter::StripRSSAd
- module: Filter::EntryFullText::SiteInfo
config:
impersonate: 0
force_upgrade: 1
- module: Store::Fastladder
config:
sync_rate: 1
connect_info:
- dbi:mysql:fastladder_production
- root
- on_connect_do:
- SET NAMES utf8
member_id: 1
Plagger実行。
$ plagger -c Sites/plagger/fastladder-crawler.yaml
Plagger [info] plugin Plagger::Plugin::Subscription::LivedoorReader loaded.
Plagger [info] plugin Plagger::Plugin::Filter::StripRSSAd loaded.
Plagger [info] plugin Plagger::Plugin::Filter::BloglinesContentNormalize loaded.
Can't locate Web/Scraper.pm in @INC (@INC contains: /opt/local/bin/lib /Users/Madhat/Sites/plagger/plagger/lib /opt/local/lib/perl5/5.8.8/darwin-2level /opt/local/lib/perl5/5.8.8 /opt/local/lib/perl5/site_perl/5.8.8/darwin-2level /opt/local/lib/perl5/site_perl/5.8.8 /opt/local/lib/perl5/site_perl /opt/local/lib/perl5/vendor_perl/5.8.8/darwin-2level /opt/local/lib/perl5/vendor_perl/5.8.8 /opt/local/lib/perl5/vendor_perl .) at /Users/Madhat/Sites/plagger/plagger/lib/Plagger/Plugin/Filter/EntryFullText/SiteInfo.pm line 9.
BEGIN failed--compilation aborted at /Users/Madhat/Sites/plagger/plagger/lib/Plagger/Plugin/Filter/EntryFullText/SiteInfo.pm line 9.
Compilation failed in require at /Users/Madhat/Sites/plagger/plagger/lib/Plagger.pm line 234.
怒られたのでWeb::Scraper入れる。
$ sudo cpan -i Web::Scraper
再度実行
$ plagger -c Sites/plagger/fastladder-crawler.yaml
Plagger [info] plugin Plagger::Plugin::Subscription::LivedoorReader loaded.
Plagger [info] plugin Plagger::Plugin::Filter::StripRSSAd loaded.
Plagger [info] plugin Plagger::Plugin::Filter::BloglinesContentNormalize loaded.
Plagger [info] plugin Plagger::Plugin::Filter::EntryFullText::SiteInfo loaded.
Plagger::Plugin::Filter::EntryFullText::SiteInfo [debug] siteinfo: ^http://b\.hatena\.ne\.jp/entry/ id("entry-info")/div[@class="section"][1]|id("bookmarked_user")
Plagger::Plugin::Filter::EntryFullText::SiteInfo [debug] siteinfo: ^http://(feeds\.)?japan\.cnet\.com //div[contains(@class,"leaf_body")]
Plagger::Plugin::Filter::EntryFullText::SiteInfo [debug] siteinfo: ^http://www\.excite\.co\.jp/News/bit //div[@class="lh140"]
...
だららーっとsiteinfoが読み込まれてく。成功したっぽい。
トラックバック(1)
トラックバックURL: http://retlet.net/cgi-bin/mt5/mt-tb.cgi/25
retlet.net - OpenFL + Store::Fastladderで... 続きを読む
Through my examination, shopping for consumer electronics online can for sure be expensive, yet there are some tricks and tips that you can use to help you get the best products. There are generally ways to locate discount promotions that could help to make one to have the best gadgets products at the cheapest prices. Interesting blog post.
I have recently started a website, the info you offer on this site has helped me tremendously. Thanks for all of your time & work.
YouTube is world's biggest video sharing website, no one can defeat it. Every one upload video tutorials at YouTube after that obtain embed code and post anyplace.
It’s nearly impossible to uncover knowledgeable males and ladies during this topic, even so you sound like do you know what you’re discussing! Thanks
I read this post fully concerning the comparison of hottest and preceding technologies, it's awesome article.
Youre so cool! I dont suppose Ive learn something like this before. So nice to search out any person with some authentic thoughts on this subject. realy thank you for beginning this up. this web site is one thing that's needed on the internet, someone with a bit of originality. useful job for bringing something new to the internet!
I just could not depart your web site prior to suggesting that I really enjoyed the standard info an individual supply on your guests? Is going to be back frequently in order to check up on new posts
I absolutely really like your blog and uncover a great deal of your post’s to be exactly I’m searching for. can you offer guest writers to write content material to suit your needs? I wouldn’t mind writing a post or elaborating on a couple of with the subjects you write in relation to here. Once again, awesome weblog!
Thanks for your text. I would like to say that a health insurance dealer also works for the benefit of the actual coordinators of a group insurance policy. The health insurance broker is given a summary of benefits sought by someone or a group coordinator. Exactly what a broker may is hunt for individuals as well as coordinators which will best complement those demands. Then he provides his referrals and if the two of you agree, the actual broker formulates binding agreement between the two parties.
Hello! I just wanted to ask if you ever have any problems with hackers? My last blog (wordpress) was hacked and I ended up losing several weeks of hard work due to no back up. Do you have any solutions to stop hackers?
Dead written topic matter, Actually enjoyed reading by means of .
choice. Anyhow; in case you are a young driver and new to the road life, then you are able to certainly horn
One thing is always that one of the most widespread incentives for applying your cards is a cash-back or even rebate provision. Generally, you'll have access to 1-5% back on various buying. Depending on the credit card, you may get 1% again on most buying, and 5% back on expenditures made on convenience stores, filling stations, grocery stores along with 'member merchants'.
I always emailed this web site post page to all my friends, as if like to read it after that my links will too.
Hey there. I want to to inquire something…is this a wordpress weblog as we are thinking about shifting over to WP. Also did you make this theme on your personal? Thanks.
Superb post however , I was wondering if you could write a litte more on this topic? I'd be very grateful if you could elaborate a little bit more. Appreciate it!
I'm just writing to make you be aware of of the exceptional encounter my princess obtained reading your web site. She noticed a lot of issues, which included what it's like to have an ideal giving nature to have most people without hassle master a variety of complex things. You really did more than visitors' expectations. Thanks for producing those valuable, trustworthy, revealing and as well as easy tips on the topic to Evelyn.
What a funny blog! I actually enjoyed watching this funny video with my family as well as together with my colleagues.
Enjoyed examining this, extremely very good stuff, thanks .
I don¡¦t even know how I finished up right here, but I thought this submit used to be great. I do not realize who you are but certainly you are going to a famous blogger in case you are not already ;) Cheers!
I sometimes read your blog, if you can, do post new stuff more frequently :D
Appreciate it for this post, I am a big fan of this internet site would like to maintain updated.
As I web website possessor I believe the content matter here is rattling magnificent , appreciate it for your hard work. You must maintain it up forever! Very best of luck.
It was really just hard to believe. http://www.fengdao.com
high quality and also a minor more effortless to give good results about the ear. Monster Beats provides you essentially the most great tone excellent. http://www.fengdao.com
Would you be serious about exchanging hyperlinks?
Fantastic article.Really thank you! Awesome.
I sometimes read your blog, if you can, do publish new stuff more frequently :D
Hello there. I found your blog via Google whilst looking for a similar topic, your site came up. It seems good. I have bookmarked it in my google bookmarks to visit later.
you may have a terrific weblog right here! would you prefer to make some invite posts on my blog?
I am continuously browsing online for ideas that can facilitate me. Thank you!
Attractive component of content. I just stumbled upon your weblog and in accession capital to assert that I get actually enjoyed account your weblog posts. Anyway I’ll be subscribing for your augment and even I fulfillment you get entry to consistently rapidly.
Thanks alot : ) for your post. I would like to write my opinion that the price of car insurance differs a lot from one policy to another, mainly because there are so many different issues which give rise to the overall cost. By way of example, the make and model of the automobile will have an enormous bearing on the charge. A reliable outdated family auto will have a less expensive premium compared to a flashy expensive car.
My boss is also keen of YouTube comic movies, he also watch these even in office hehehe..
Howdy very nice blog!! Man .. Excellent .. Amazing .. I will bookmark your web site and take the feeds also…I am satisfied to search out so many useful information right here within the submit, we'd like develop more strategies on this regard, thanks for sharing. . . . . .
Excellent post, you might have pointed out some great details , I besides believe this s a very wonderful website.
Hong Kong these days continues to be one particular of the best offshore banking jurisdictions. It offers a fantastic mixture of bank secrecy, corporate secrecy, a financially and politically stable environment, and sturdy banks. But probably most importantly, it can be a secure offshore investment haven for those who want to diversify out of sinking western currencies into booming Asian markets, and China in particular.
Awesome site! How did you get that layout?
you might be in reality a outstanding webmaster. The website loading pace is incredible. It seems that you’re performing any distinctive trick. Also, The contents are masterpiece. you've performed a wonderful task on this subject!
I admire your work , regards for all of the beneficial weblog posts.
There a few fascinating points more than time here but I don’t know if I see them all center to heart. There exists some validity but Let me take hold opinion until I appear into it further. Really great post , thanks and now we want much more! Included with FeedBurner at the same time
the actual plan lines are quite dull.
Thanks for the suggestions you might have provided here. 1 more thing I would like to mention is that laptop memory requirements normally increase along with other breakthroughs inside the engineering. For instance, as soon as new generations of processors are introduced towards the market, there is typically a matching increase inside the shape demands of all computer system memory plus hard drive room. This really is because the application operated by these cpus will inevitably boost in power to leverage the new engineering.
I am genuinely experiencing the design and design of the internet site. This is a a breeze for the eye that makes it considerably more enjoyable that i can occur below along with pay a visit to more regularly. Did you hire out there a new creator to produce your own design? Fantastic perform!
This is the right blog for anyone who wants to find out about this topic. You realize so much its almost hard to argue with you (not that I actually would want…HaHa). You definitely put a new spin on a topic thats been written about for years. Great stuff, just great!
I love reading and I believe this website got some truly useful stuff on it!
Hi! I know this is kind of off topic but I was wondering which blog platform are you using for this website? I'm getting sick and tired of Wordpress because I've had problems with hackers and I'm looking at alternatives for another platform. I would be awesome if you could point me in the direction of a good platform.
I’m truly enjoying the style and layout of your internet site. It is a quite simple on the eyes which makes it significantly a lot more enjoyable for me to come here and go to more often. Did you hire out a designer to create your theme? Exceptional function!
I'm often to blogging and i actually appreciate your content. The article has actually peaks my interest. I am going to bookmark your website and hold checking for brand new information.
Can I just now say what a relief to seek out 1 who in fact knows what theyre dealing with on-line. You really recognize how to bring a concern to light and make it essential. Lots far more individuals need to have to see this and understand why side within the story. I cant believe youre less common since you also definitely hold the gift.