ubuntu|编译安装mysql2pgsql

dbsync 项目目标是围绕 PostgreSQL Greenplum ,实现易用的数据的互迁功能。
官方地址:
https://github.com/aliyun/rds_dbsync
提供mysql2pgsql和pgsql2pgsql两个工具.
不过竟然只提供了Centos版或者说rpm版.而我司服务器清一色ubuntu,不能因为这个工具,存在一个异类吧?于是开启了这趟不大顺利的编译之旅

系统

Distributor ID: Ubuntu
Description:    Ubuntu 16.04.6 LTS
Release:        16.04
Codename:       xenial

下载代码

放在src目录是了习惯

cd /usr/local/src
git clone https://github.com/aliyun/rds_dbsync.git

安装mysql的开发包

既然编译安装自然依赖mysql的开发包

apt-get install  mysql-community-embedded-devel

安装pg的开发包

pg的包安装相对麻烦一些,毕竟不常用

echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add
apt-get update

apt-get install postgresql-server-dev-9.6
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  comerr-dev krb5-multidev libgssrpc4 libkadm5clnt-mit9 libkadm5srv-mit9 libkdb5-8
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libpq-dev libpq5 postgresql-client-9.6
Suggested packages:
  postgresql-doc-12 postgresql-9.6 postgresql-doc-9.6
The following NEW packages will be installed:
  postgresql-client-9.6 postgresql-server-dev-9.6
The following packages will be upgraded:
  libpq-dev libpq5
2 upgraded, 2 newly installed, 0 to remove and 52 not upgraded.
Need to get 2,438 kB of archives.
After this operation, 10.8 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main amd64 libpq-dev amd64 12.3-1.pgdg16.04+1 [139 kB]
Get:2 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main amd64 libpq5 amd64 12.3-1.pgdg16.04+1 [172 kB]                                   
Get:3 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main amd64 postgresql-client-9.6 amd64 9.6.18-1.pgdg16.04+1 [1,343 kB]                
Get:4 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main amd64 postgresql-server-dev-9.6 amd64 9.6.18-1.pgdg16.04+1 [784 kB]              
Fetched 2,438 kB in 4min 24s (9,223 B/s)                                                                                                        
(Reading database ... 105399 files and directories currently installed.)
Preparing to unpack .../libpq-dev_12.3-1.pgdg16.04+1_amd64.deb ...
Unpacking libpq-dev (12.3-1.pgdg16.04+1) over (9.5.21-0ubuntu0.16.04.1) ...
Preparing to unpack .../libpq5_12.3-1.pgdg16.04+1_amd64.deb ...
Unpacking libpq5:amd64 (12.3-1.pgdg16.04+1) over (9.5.21-0ubuntu0.16.04.1) ...
Selecting previously unselected package postgresql-client-9.6.
Preparing to unpack .../postgresql-client-9.6_9.6.18-1.pgdg16.04+1_amd64.deb ...
Unpacking postgresql-client-9.6 (9.6.18-1.pgdg16.04+1) ...
Selecting previously unselected package postgresql-server-dev-9.6.
Preparing to unpack .../postgresql-server-dev-9.6_9.6.18-1.pgdg16.04+1_amd64.deb ...
Unpacking postgresql-server-dev-9.6 (9.6.18-1.pgdg16.04+1) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for libc-bin (2.23-0ubuntu11) ...
Processing triggers for postgresql-common (173ubuntu0.3) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Setting up libpq5:amd64 (12.3-1.pgdg16.04+1) ...
Setting up libpq-dev (12.3-1.pgdg16.04+1) ...
Setting up postgresql-client-9.6 (9.6.18-1.pgdg16.04+1) ...
update-alternatives: using /usr/share/postgresql/9.6/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode
Setting up postgresql-server-dev-9.6 (9.6.18-1.pgdg16.04+1) ...
Processing triggers for libc-bin (2.23-0ubuntu11) ...

编译

尴尬开始

cd rds_dbsync/dbsync
make
/usr/lib/postgresql/9.6/lib/pgxs/src/makefiles/pgxs.mk:219: warning: ignoring old recipe for target 'clean'
g++ -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer -fPIC -fPIC demo.o pg_logicaldecode.o pqformat.o stringinfo.o utils.o misc.o pgsync.o ini.o -L/usr/lib/postgresql/9.6/lib -lpgcommon -lpgport -L/usr/lib/x86_64-linux-gnu -lpq '-Wl,-rpath,$ORIGIN,-rpath,$ORIGIN/lib,-rpath,$ORIGIN/../lib,-rpath,/usr/lib/x86_64-linux-gnu,-rpath,/usr/lib/x86_64-linux-gnu' -L/usr/lib/x86_64-linux-gnu  -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now  -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,--as-needed  -lpthread -o demo 
/usr/bin/ld: demo.o: relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
demo.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:27: recipe for target 'all' failed
make: *** [all] Error 1

网上查到两个同类问题

问题:fileappender.o relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPI
解决:
使用 ar -x liblog4cplus.a 解包
readelf --relocs fileappender.o | egrep '(GOT|PLT|JU?MP_SLOT)' 如果输出为空,则该静态库没有position-independent,需要在该静态库的编译脚本添加 -fPIC

ffmpeg  --enable-shared编译时
libavcodec/mqc.o: relocation r_x86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fpic
ffmpeg编译错误,修改Makefile
121:$$(OBJS-$(1)): CFLAGS  += $(CFLAGS-$(1))
为
121:$$(OBJS-$(1)): CFLAGS  += -fPIC $(CFLAGS-$(1))
重新make,成功。

解决办法都是添加-fPIC
经过一番尝试,发现其实编译过程已经添加了-fPIC,但是始终编译不过
查看demo.cpp发现,里面竟然写死了数据库连接的用户名和密码

看来是测试的demo了,那不是可以直接不编译这哥们儿?
将Makefile中demo相关的代码注释掉

-all: demo.o dbsync-pgsql2pgsql.o mysql2pgsql.o dbsync-mysql2pgsql.o readcfg.o
-       $(CXX) $(CFLAGS) demo.o $(OBJS) $(libpq_pgport) $(RPATH_LDFLAGS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o demo 
+#all: demo.o dbsync-pgsql2pgsql.o mysql2pgsql.o dbsync-mysql2pgsql.o readcfg.o
+all: dbsync-pgsql2pgsql.o mysql2pgsql.o dbsync-mysql2pgsql.o readcfg.o
+       #$(CXX) $(CFLAGS) demo.o $(OBJS) $(libpq_pgport) $(RPATH_LDFLAGS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o demo 
        $(CXX) $(CFLAGS) readcfg.o dbsync-pgsql2pgsql.o $(OBJS) $(libpq_pgport) $(RPATH_LDFLAGS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o pgsql2pgsql
        $(CXX) $(CFLAGS) readcfg.o ini.o mysql2pgsql.o dbsync-mysql2pgsql.o misc.o stringinfo.o $(libpq_pgport) $(RPATH_LDFLAGS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -L$(mysql_lib_dir) -lmysqlclient -o mysql2pgsql

@@ -36,7 +37,7 @@ package:
        mkdir -p install/bin
        mkdir -p install/lib
        cp -fr pgsql2pgsql install/bin
-       cp -fr demo install/bin
+#      cp -fr demo install/bin
        cp -fr ali_recvlogical.so install/lib
        cp -fr mysql2pgsql install/bin
        cp -fr $(mysql_lib_dir)/libmysqlclient.so* install/lib

再次make,顺利通过

打包二进制

# make package 

mkdir -p install
mkdir -p install/bin
mkdir -p install/lib
cp -fr pgsql2pgsql install/bin
cp -fr ali_recvlogical.so install/lib
cp -fr mysql2pgsql install/bin
cp -fr /usr/lib/x86_64-linux-gnu/libmysqlclient.so* install/lib
cp -fr /usr/lib/x86_64-linux-gnu/libpq.so* install/lib

将生成一个install目录,里面有二进制和lib

验证

#cd install/bin/
#ls
mysql2pgsql  pgsql2pgsql
#./mysql2pgsql 
Failed to load config file
read config file error, insufficient permissions or my.cfg does not exist

看起来工作ok

总结

编译过程看似复杂,其实就卡在一个测试的demo文件上.花了有两个小时研究解决方案.好尴尬.

转载请注明: 转自Rainbird的个人博客
   本文链接: ubuntu|编译安装mysql2pgsql


相关博文

About rainbird

IOS攻城狮
This entry was posted in linux and tagged , , , , , , , , , , , , . Bookmark the permalink.

发表评论