OpenFL + MySQL + Plagger::Plugin::Store::Fastladder 入れた

MySQLのインストール

$ sudo port install mysql5 +setver
--->  Verifying checksum(s) for mysql5
Error: Checksum (md5) mismatch for mysql-5.0.51.tar.gz
Error: Checksum (sha1) mismatch for mysql-5.0.51.tar.gz
Error: Checksum (rmd160) mismatch for mysql-5.0.51.tar.gz
Error: Target org.macports.checksum returned: Unable to verify file checksums

いきなり転けた＞＜

$ sudo port sync

とかしてたら出来るようになった。

$ sudo port install mysql5 +server
--->  Verifying checksum(s) for mysql5
Error: Target org.macports.checksum returned: Could not open file: /opt/local/var/macports/distfiles/mysql5/mysql-5.0.51a.tar.gz
Error: Status 1 encountered during processing.

ひゃー。
エラーでググったらクリーンすれば良さそう。

$ sudo port clean --all mysql5
--->  Cleaning mysql5

今度こそ。

$ sudo port install mysql5 +server
--->  Fetching mysql5
--->  Attempting to fetch mysql-5.0.51a.tar.gz from http://mysql.mirrors.pair.com/Downloads/MySQL-5.0/
--->  Verifying checksum(s) for mysql5
--->  Extracting mysql5
--->  Configuring mysql5
--->  Building mysql5 with target all
--->  Staging mysql5 into destroot
--->  Creating launchd control script
###########################################################
# A startup item has been generated that will aid in
# starting mysql5 with launchd. It is disabled
# by default. Execute the following command to start it,
# and to cause it to launch at startup:
#
# sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist
###########################################################
--->  Installing mysql5 5.0.51a_0+server
******************************************************
* In order to setup the database, you might want to run
* sudo -u mysql mysql_install_db5
* if this is a new install
******************************************************
--->  Activating mysql5 5.0.51a_0+server
--->  Cleaning mysql5

できた。
あとはしゅにたんの記事の通りに。

OpenFLのDB設定

まずOpenFLの最新版を入手。以前インストールしたときはsvnを使わなかったので、ディレクトリ名指定で上書きしてみる。

$ cd ~/Sites/openfl
$ svn co http://fastladder.googlecode.com/svn/trunk/ fastladder

database.ymlをMySQL用にしてproduction項を編集。

$ cd fastladder
$ cp config/database.yml.mysql config/database.yml
$ vim config/database.yml
...
production:
  adapter: mysql
  encoding: utf8
  database: fastladder_production
  username: root
  password:
  socket: /opt/local/var/run/mysql5/mysqld.sock

DBを初期化。

$ mysqladmin5 -u root create fastladder_production
$ RAILS_ENV=production rake db:migrate

起動。

$ script/server -e -d production
=> Booting Mongrel (use 'script/server webrick' to force WEBrick)
=> Rails application starting on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
** Starting Mongrel listening at 0.0.0.0:3000
** Starting Rails with -d environment...
Exiting
/opt/local/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/initializer.rb:206:in `read': No such file or directory - /Users/Madhat/Sites/openfl/fastladder/config/environments/-d.rb (Errno::ENOENT)
        from /opt/local/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/initializer.rb:206:in `load_environment'
        from /opt/local/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/core_ext/kernel/reporting.rb:11:in `silence_warnings'
        from /opt/local/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/initializer.rb:199:in `load_environment'
        from /opt/local/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/initializer.rb:91:in `process'
        from /opt/local/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/initializer.rb:49:in `send'
        from /opt/local/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/initializer.rb:49:in `run'
        from /Users/Madhat/Sites/openfl/fastladder/config/environment.rb:13
        from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
         ... 25 levels...
        from /opt/local/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/commands/server.rb:39
        from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
        from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `require'
        from script/server:3

なんぞー。
って、-e と -d が逆？

$ script/server -d -e production
=> Booting Mongrel (use 'script/server webrick' to force WEBrick)
=> Rails application starting on http://0.0.0.0:3000

動いた。しゅにたんめ( ﾟ皿ﾟ)
http://0.0.0.0:3000/ にアクセスしてアカウントを作成。

DBD:mysqlのインストール

$ sudo cpan -i DBD::mysql
...
Can't locate DBI/DBD.pm in @INC ...

DBI::DBDが無いって怒られたので先にインストール。

$ sudo cpan -i DBI::DBD

ワンモア。

$ sudo cpan -i DBD::mysql
...
Running make test
  Make had some problems, maybe interrupted? Won't test
Running make install
  Make had some problems, maybe interrupted? Won't install

MySQLのコンフィグにシンボリックリンクを張る。

sudo ln -s /opt/local/lib/mysql5/bin/mysql_config /usr/local/bin/mysql_config

cpanの作業ディレクトリに移動。と思ったんだけど DBD-mysql-.*/ が無い！
......って、場所が違った罠。

$ cd /var/root/.cpan/build/DBD-mysql-4.007
-bash: cd: /var/root/.cpan/build/DBD-mysql-4.007: Permission denied

パーミッションで怒られた。

$ su root
Password:
su: Sorry

なんでだっけ。
しょうがないので一時的に /var/root/ のアクセス権変えた。

$ cd /var/root/.cpan
$ sudo perl Makefile.PL --cflags=-I/opt/local/include/mysql5/mysql "--libs=-L/opt/local/lib -L/opt/local/lib/mysql5/mysql -lmysqlclient -L/opt/local/lib -lz -lm  -L/opt/local/lib -lssl -lcrypto"
...
Writing Makefile for DBD::mysql

準備完了。makeする。

$ sudo make
$ sudo make test
$ sudo make install

Store::Fastladderのインストール

$ cd ~/Sites/openfl/
$ svn co http://svn.bulknews.net/repos/plagger/branches/fastladder-crawler/plagger/ fastladder-crawler

Plaggerのディレクトリにエイリアスを作成。

$ ln -s fastladder-crawler/lib/Plagger/Plugin/Store/ ~/Sites/plagger/cpan/lib/perl5/site_perl/Plagger/Plugin/Store

fastladder-crawler.yaml を作成。

global:
  timezone: Asia/Tokyo
  log:
    level: debug

plugins:
  - module: Subscription::Config
    config:
      feed:
        - url: http://b.hatena.ne.jp/retlet/atomfeed

  - module: Store::Fastladder
    config:
      connect_info:
        - dbi:mysql:fastladder_production
        - root
        - on_connect_do:
            - SET NAMES utf8
      member_id: 1

Plaggerでクローラーを動かす

Plagger実行。

$ cd ~/Sites/plagger
$ bin/plagger -c fastladder-crawler.yaml 
Plagger [info] plugin Plagger::Plugin::Subscription::Config loaded.
Base class package "DBIx::Class::Schema::Loader" is empty.
    (Perhaps you need to 'use' the module which defines that package first.)
 at /Users/Madhat/Sites/plagger/cpan/lib/perl5/site_perl/Plagger/Plugin/Store/Fastladder/Schema/Loader.pm line 10
BEGIN failed--compilation aborted at /Users/Madhat/Sites/plagger/cpan/lib/perl5/site_perl/Plagger/Plugin/Store/Fastladder/Schema/Loader.pm line 10, <DATA> line 1.
Compilation failed in require at /Users/Madhat/Sites/plagger/cpan/lib/perl5/site_perl/Plagger/Plugin/Store/Fastladder.pm line 11, <DATA> line 1.
BEGIN failed--compilation aborted at /Users/Madhat/Sites/plagger/cpan/lib/perl5/site_perl/Plagger/Plugin/Store/Fastladder.pm line 11, <DATA> line 1.
Compilation failed in require at /Users/Madhat/Sites/plagger/cpan/lib/perl5/site_perl/Plagger.pm line 232, <DATA> line 1.

＞＜

$ sudo cpan -i DBIx::Class::Schema::Loader
...
  /usr/bin/make install  -- OK

質問は全部Enter連打した。
って、やっぱり同じエラー出る。なんぞ。

とりあえずPlaggerが古いので最新版にしてみる。

$ cd ../plagger
$ svn co http://svn.bulknews.net/repos/plagger/trunk/plagger
$ cd plagger/
$ perl Makefile.PL
Can't locate inc/Module/Install.pm in @INC...

モジュール足りないので入れる。

$ sudo cpan -i inc::Module::Install
...[Enter]連打

どうや。

$ perl Makefile.PL 
include /Users/Madhat/Sites/plagger/plagger/inc/Module/Install.pm
Bareword "use_test_base" not allowed while "strict subs" in use at Makefile.PL line 114.
Execution of Makefile.PL aborted due to compilation errors.

むむ。PlaggerFAQ - Plagger - Trac見ると、YAMLとModule::InstallとTest::Baseを入れろとあった。

$ sudo cpan -i YAML
$ sudo cpan -i Module::Install
$ sudo cpan -i Test::Base

で今度こそ。

$ perl Makefile.PL
...[Enter]連打
Writing Makefile for Plagger

できた。makeする。

$ sudo make
$ sudo make test
...
3 tests skipped.
Failed 210/214 test scripts, 1.87% okay. 4/263 subtests failed, 98.48% okay.
make: *** [test_dynamic] Error 2

うわーん。

Can't locate XML/Atom.pm in @INC

が目についたので入れてみる。

$ sudo cpan -i XML::Atom
...
Failed 3/20 test scripts, 85.00% okay. 3/278 subtests failed, 98.92% okay.
make: *** [test_dynamic] Error 255
  /usr/bin/make test -- NOT OK
Running make install
  make test had returned bad status, won't install without force

うーん。こういうときはどの辺を見れば原因がわかるのかな。
無理矢理やってしまおう。

$ sudo cpan
cpan[1]> force install XML::Feed

(cpan> test Plaggerして不足モジュールをチェックしたところ、XML::Feedも足りなかったのでXML::Feedから入れた。XML::Atomも一緒に入ったので次のステップへ)

make testしようと思ったけど、ssig33に「testなんか気にすんな」と言われたので気にせずインストール。

$ sudo make install
...
Writing /opt/local/lib/perl5/site_perl/5.8.8/darwin-2level/auto/Plagger/.packlist
Appending installation info to /opt/local/lib/perl5/5.8.8/darwin-2level/perllocal.pod

成功っぽい。

$ plagger -c ../fastladder-crawler.yaml
...
Plagger [info] plugin Plagger::Plugin::Subscription::Config loaded.
Can't locate Plagger/Plugin/Store/Fastladder.pm in @INC (@INC contains: /opt/local/bin/lib /sw/lib/perl5 /sw/lib/perl5/darwin /opt/local/lib/perl5/5.8.8/darwin-2level /opt/local/lib/perl5/5.8.8 /opt/local/lib/perl5/site_perl/5.8.8/darwin-2level /opt/local/lib/perl5/site_perl/5.8.8 /opt/local/lib/perl5/site_perl /opt/local/lib/perl5/vendor_perl/5.8.8/darwin-2level /opt/local/lib/perl5/vendor_perl/5.8.8 /opt/local/lib/perl5/vendor_perl .) at /opt/local/lib/perl5/site_perl/5.8.8/Plagger.pm line 234, <DATA> line 1.

エイリアス作るの忘れてたのでコピーしてもう一度。

$ ln -s fastladder-crawler/lib/Plagger/Plugin/Store/ ~/Sites/plagger/plagger/lib/Plugin/Store
$ plagger -c ../fastladder-crawler.yaml 
Plagger [info] plugin Plagger::Plugin::Subscription::Config loaded.
...

同じエラー。よく見たらパスが通ってない。

$ vim ~/.bashrc
...
export PERL5LIB=~/Sites/plagger/plagger/lib/
...
$ source ~/.bashrc

$ plagger -c ../fastladder-crawler.yaml 
Plagger [info] plugin Plagger::Plugin::Subscription::Config loaded.
Plagger [info] plugin Plagger::Plugin::Store::Fastladder loaded.
DBIx::Class::Storage::DBI::datetime_parser(): Couldn't load DateTime::Format::MySQL: Can't locate DateTime/Format/MySQL.pm in @INC (@INC contains: /opt/local/bin/lib /Users/Madhat/Sites/plagger/plagger/lib /opt/local/lib/perl5/5.8.8/darwin-2level /opt/local/lib/perl5/5.8.8 /opt/local/lib/perl5/site_perl/5.8.8/darwin-2level /opt/local/lib/perl5/site_perl/5.8.8 /opt/local/lib/perl5/site_perl /opt/local/lib/perl5/vendor_perl/5.8.8/darwin-2level /opt/local/lib/perl5/vendor_perl/5.8.8 /opt/local/lib/perl5/vendor_perl .) at (eval 1048) line 2, <DATA> line 1.
BEGIN failed--compilation aborted at (eval 1048) line 2, <DATA> line 1.

一歩前進。
モジュール入れる。

$ sudo cpan -i DateTime::Format::MySQL

こーれーでーどーだー。

$ plagger -c ../fastladder-crawler.yaml 
Plagger [info] plugin Plagger::Plugin::Subscription::Config loaded.
Plagger [info] plugin Plagger::Plugin::Store::Fastladder loaded.
Plagger [info] plugin Plagger::Plugin::Bundle::Defaults loaded.
Plagger [info] plugin Plagger::Plugin::Aggregator::Simple loaded.
Plagger [info] plugin Plagger::Plugin::Summary::Auto loaded.
Plagger [info] plugin Plagger::Plugin::Summary::Simple loaded.
Plagger [info] plugin Plagger::Plugin::Namespace::HatenaFotolife loaded.
Plagger [info] plugin Plagger::Plugin::Namespace::MediaRSS loaded.
Plagger [info] plugin Plagger::Plugin::Namespace::ApplePhotocast loaded.
Plagger::Plugin::Aggregator::Simple [info] Fetch http://b.hatena.ne.jp/retlet/atomfeed
Plagger::Cache [debug] Cache MISS: Aggregator-Simple|http://b.hatena.ne.jp/retlet/atomfeed
Plagger::Plugin::Aggregator::Simple [debug] 200: http://b.hatena.ne.jp/retlet/atomfeed
Plagger::Plugin::Aggregator::Simple [info] Aggregate http://b.hatena.ne.jp/retlet/atomfeed success: 30 entries.

できたー！！
おめでとうございます。ありがとうございます。