{"id":4839,"date":"2020-02-28T18:16:23","date_gmt":"2020-02-28T10:16:23","guid":{"rendered":"https:\/\/www.xttblog.com\/?p=4839"},"modified":"2020-02-28T18:16:23","modified_gmt":"2020-02-28T10:16:23","slug":"canal-adapter%e8%b6%9f%e5%9d%91%e8%ae%b0%e5%bd%95","status":"publish","type":"post","link":"https:\/\/www.xttblog.com\/?p=4839","title":{"rendered":"canal-adapter\u8d9f\u5751\u8bb0\u5f55"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">\u7528\u8fc7 canal \u7684\u90fd\u77e5\u9053\uff0ccanal \u73b0\u5728\u5751\u975e\u5e38\u7684\u591a\uff0c\u672c\u6587\u8bb0\u5f55\u4e00\u4e2a\u6211\u9047\u5230\u7684\u5751\u4e4b\u4e00\uff01<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">canal-adapter\u76ee\u524d\u652f\u6301rdb\u3001es\u3001kafka\u3001hbase\u7b49\u591a\u4e2a\u76ee\u6807\u7aef\u7684\u540c\u6b65\uff0c\u6700\u8fd1\u51e0\u7bc7\u6211\u4f1a\u8bb0\u5f55\u4e00\u4e0b\u6211\u5728\u4f7f\u7528adapter\u5411\u8fd9\u4e9b\u76ee\u6807\u7aef\u540c\u6b65\u65f6\uff0c\u662f\u5982\u4f55\u89e3\u51b3es\u3001hbase\u7248\u672c\u9002\u914d\uff0c\u6dfb\u52a0\u90e8\u5206\u4e2a\u6027\u5316\u9700\u6c42\uff0c\u4ee5\u53ca\u5982\u4f55\u5904\u7406\u4e00\u4e9b\u6211\u9047\u5230\u5404\u79cd\u95ee\u9898\u3002\u4eca\u5929\u5148\u6765\u8bb0\u5f55\u4e00\u4e0badapter\u540c\u6b65mysql\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u540c\u6b65mysql<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">application.yml \u914d\u7f6e\u6587\u4ef6\u5185\u5bb9\u5982\u4e0b\u6240\u793a\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>server:\n  port: 8081\nspring:\n  jackson:\n    date-format: yyyy-MM-dd HH:mm:ss\n    time-zone: GMT+8\n    default-property-inclusion: non_null\n\ncanal.conf:\n  mode: tcp # kafka rocketMQ\n  canalServerHost: 127.0.0.1:11111\n# zookeeperHosts: slave1:2181\n# mqServers: 127.0.0.1:9092 #or rocketmq\n# flatMessage: true\n  batchSize: 500\n  syncBatchSize: 1000\n  retries: 0\n  timeout:\n  accessKey:\n  secretKey:\n  username:\n  password:\n  vhost:\n  srcDataSources:\n    defaultDS:\n      url: jdbc:mysql:\/\/127.0.0.1:3306\/mytest?useUnicode=true\n      username: root\n      password: 121212\n  canalAdapters:\n  - instance: example # canal instance Name or mq topic name\n    groups:\n    - groupId: g1\n      outerAdapters:\n      - name: rdb\n      # \u5728\u6240\u6709\u76ee\u6807\u7aef\u7684\u540c\u6b65\u4e2dkey\u662f\u4e00\u4e2a\u5fc5\u987b\u8981\u914d\u7f6e\u7684\u53c2\u6570\uff0c\u4e0d\u914d\u7f6e\u53ef\u80fd\u4f1a\u51fa\u73b0\u6570\u636e\u4e22\u5931\u7684\u95ee\u9898\n        key: mysql1\n        properties:\n          jdbc.driverClassName: com.mysql.jdbc.Driver\n          jdbc.url: jdbc:mysql:\/\/127.0.0.1:3306\/mytest2?useUnicode=true\n          jdbc.username: root\n          jdbc.password: 121212<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">conf\/rdb\u4e0b\u914d\u7f6e\u540c\u6b65\u8868\u4fe1\u606f\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>dataSourceKey: defaultDS\ndestination: example\ngroupId: ''\n# \u5728\u6240\u6709\u76ee\u6807\u7aef\u7684\u540c\u6b65\u4e2douterAdapterKey\u662f\u4e00\u4e2a\u5fc5\u987b\u8981\u914d\u7f6e\u7684\u53c2\u6570\uff0c\u4e0d\u914d\u7f6e\u53ef\u80fd\u4f1a\u51fa\u73b0\u6570\u636e\u4e22\u5931\u7684\u95ee\u9898\n# \u5bf9\u5e94application.yml\u4e2dkey\u5b57\u6bb5\nouterAdapterKey: mysql1\nconcurrent: false\ndbMapping:\n  commitBatch: 5000\n  etlCondition: null\n  mirrorDb: false\n  readBatch: 5000\n  database: mytest # \u6e90\u6570\u636e\u6e90\u7684database\/shcema\n  table: tb1 # \u6e90\u6570\u636e\u6e90\u8868\u540d\n  targetTable: mytest2.tb1 # \u76ee\u6807\u6570\u636e\u6e90\u7684\u5e93\u540d.\u8868\u540d\n  targetPk: # \u4e3b\u952e\u6620\u5c04\n    ID: ID # \u5982\u679c\u662f\u590d\u5408\u4e3b\u952e\u53ef\u4ee5\u6362\u884c\u6620\u5c04\u591a\u4e2a\n  mapAll: true<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">\u95ee\u9898<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">\u5728\u540c\u6b65\u8fc7\u7a0b\u4e2d\uff0c\u51fa\u73b0\u4e86insert\u6210\u529f\uff0cdelete\u548cupdate\u65e5\u5fd7\u53cd\u9988\u6210\u529f\uff0c\u4f46\u5b9e\u9645\u6570\u636e\u6ca1\u6709\u540c\u6b65\u7684\u73b0\u8c61\uff0c\u67e5\u770b\u540c\u6b65\u6e90\u7801\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">com.alibaba.otter.canal.client.adapter.rdb.service.RdbSyncService.java<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for (Map.Entry&lt;String, String&gt; entry : columnsMap.entrySet()) {\n            \n    \/\/\u9996\u5148\u6ce8\u610f\u6620\u5c04\u5b57\u6bb5\u5192\u53f7\u524d\u662f\u76ee\u6807\u5b57\u6bb5\u540d\uff0c\u5192\u53f7\u540e\u662f\u6e90\u7aef\u5b57\u6bb5\u540d\n    String targetColumnName = entry.getKey();\n    String srcColumnName = entry.getValue();\n    \n    \/\/\u5982\u679c\u672a\u914d\u7f6e\u5192\u53f7\u540e\u5b57\u6bb5\uff0c\u5c06\u5176\u5904\u7406\u6210\u4e0e\u5192\u53f7\u524d\u4e00\u81f4\n    if (srcColumnName == null) {\n        srcColumnName = Util.cleanColumn(targetColumnName);\n    }\n\n    Integer type = ctype.get(Util.cleanColumn(targetColumnName).toLowerCase());\n    if (type == null) {\n        throw new RuntimeException(\"Target column: \" + targetColumnName + \" not matched\");\n    }\n    \n    \/\/\u4e3b\u8981\u95ee\u9898\u51fa\u5728\u8fd9\u91cc\n    Object value = data.get(srcColumnName);\n    BatchExecutor.setValue(values, type, value);\n}<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u8fd9\u6bb5\u4ee3\u7801\uff0c\u80fd\u591f\u770b\u51fa\uff0c\u540c\u6b65\u8fc7\u7a0b\u4e2d\u66f4\u65b0\u7684value\u503c\u662f\u5b58\u50a8\u5728data\u8fd9\u4e2aMap\u7ed3\u6784\u4e2d\u7684\uff0c\u6240\u4ee5\u8fd9\u5c31\u8981\u6c42srcColumnName\u8fd9\u4e2a\u5b57\u6bb5\u7684\u503c\u8981\u4e0e\u6e90\u7aef\u6570\u636e\u8868\u7684\u5b57\u6bb5\u540d\u5927\u5c0f\u5199\u5b8c\u5168\u4e00\u81f4\uff08\u5373\u4f7f\u6570\u636e\u5e93\u672c\u8eab\u5bf9\u4e8e\u5b57\u6bb5\u5927\u5c0f\u5199\u662f\u4e0d\u654f\u611f\u7684\uff09<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u4e3e\u4f8b<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">\u6e90\u7aeftb\u6709ID\u3001id1\u3001Id2\u4e09\u5217<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u76ee\u6807\u7aeftb2\u6709id\u3001ID1\u3001id2\u4e09\u5217<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5176\u5b9e\u5728\u6570\u636e\u67e5\u8be2\u8fc7\u7a0b\u4e2d\uff0c\u7531\u4e8e\u6570\u636e\u5e93\u5bf9\u5b57\u6bb5\u5927\u5c0f\u5199\u4e0d\u654f\u611f\uff0c\u6240\u6709\u65e0\u8bba\u600e\u4e48\u5199\u90fd\u662f\u6ca1\u95ee\u9898\u7684\uff0c\u4f46\u5728\u914d\u7f6e\u540c\u6b65\u8868\u8fc7\u7a0b\u4e2d\uff0c\u4ed6\u4eec\u7684\u8868\u73b0\u5c31\u6709\u6240\u4e0d\u540c\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>targetPk: # \u4e3b\u952e\u6620\u5c04\n    ID: ID\n    \ntargetPk: # \u4e3b\u952e\u6620\u5c04\n ID:\n  \ntargetPk: # \u4e3b\u952e\u6620\u5c04\n id: ID\n    \n#\u4ee5\u4e0a\u51e0\u79cd\u5199\u6cd5\uff0c\u6700\u540e\u89e3\u6790\u51fa\u6765\u7684srcColumnName \u5747\u4e3aID\uff0c\u6240\u4ee5\u53ef\u4ee5\u4ece\u6709binlog\u89e3\u6790\u51fa\u6765\u7684\u6570\u636e\u8f6c\u6362\u5f97\u5230\u7684\u6e90\u7aefMap\u7c7b\u578b\u7684data\u4e2d\u83b7\u53d6\u5230\u6570\u636e<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>targetPk: # \u4e3b\u952e\u6620\u5c04\n  id:\n#\u7c7b\u4f3c\u8fd9\u79cd\u5199\u6cd5\uff0c\u89e3\u6790\u51fa\u7684srcColumnName \u5747\u4e3aid\uff0c\n#\u800cMap\u7c7b\u578b\u4e2ddata\u4e2d\u53ea\u5305\u542bID\u8fd9\u4e2akey\uff0c\u800c\u4e0d\u5305\u542bid\u8fd9\u4e2akey\uff0c\n#\u6240\u4ee5\u5c31\u53d6\u4e0d\u5230\u76f8\u5e94\u7684\u503c\uff0c\u5bfc\u81f4\u9700\u8981\u4e3b\u952e\u64cd\u4f5c\u7684update\u548cdelete\u8bed\u53e5\u6267\u884c\u65e0\u6548<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u8fd9\u662f\u6211\u5728\u540c\u6b65rdb\u65f6\u9047\u5230\u7684\u9700\u8981\u6ce8\u610f\u7684\u5730\u65b9\uff0c\u5f53\u7136\u4e5f\u53ef\u4ee5\u6839\u636e\u9700\u8981\u5c06\u8fd9\u90e8\u5206\u4ee3\u7801\u6539\u9020\u4e00\u4e0b\uff0c\u8fd9\u91cc\u5c31\u4e0d\u4f5c\u63cf\u8ff0\u4e86\uff0c\u4ee3\u7801\u5177\u4f53\u4f4d\u7f6e\u5df2\u7ed9\u51fa\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5982\u679c\u4f60\u4e5f\u9047\u5230\u4e86\u4e00\u4e9b\u5751\uff0c\u6b22\u8fce\u52a0\u6211\u5fae\u4fe1\u53f7\uff1acanal\uff0c\u62c9\u4f60\u8fdb\u7fa4\u5b66 canal\u3002\u63a8\u8350\u9605\u8bfb\uff1ahttps:\/\/mp.weixin.qq.com\/s\/Qerwilr7D8HDdzd1-jd_4w<\/p>\n<!-- <audio autoplay=\"autoplay\" height=\"100\" width=\"100\" hidden=\"true\"><source src=\"https:\/\/c.coka.la\/orAvPR.mp3\" type=\"audio\/mp3\"\/><\/audio><img decoding=\"async\" src=\"https:\/\/images.weserv.nl\/?url=https:\/\/ws3.sinaimg.cn\/large\/8fa5dcfcgy1g4em3rzar5j20ib089ae0.jpg\" alt=\"\u4e1a\u4f59\u8349\u516c\u4f17\u53f7\"\/> --><p><!-- <img decoding=\"async\" src=\"https:\/\/raw.githubusercontent.com\/xmt1139057136\/xttblog\/master\/8fa5dcfcgy1g4em3rzar5j20ib089ae0.jpg\" alt=\"\u4e1a\u4f59\u8349\u516c\u4f17\u53f7\"\/>--><img decoding=\"async\" src=\"https:\/\/saas.51autoparts.cn\/api\/admin\/sys-file\/oss\/file?fileName=84db51f8561e448f9a4fda2eff0811f9.jpg\" alt=\"\u4e1a\u4f59\u8349\u516c\u4f17\u53f7\"\/><\/p><p>\u6700\u540e\uff0c\u6b22\u8fce\u5173\u6ce8\u6211\u7684\u4e2a\u4eba\u5fae\u4fe1\u516c\u4f17\u53f7\uff1a\u4e1a\u4f59\u8349\uff08yyucao\uff09\uff01<b>\u53ef\u52a0\u4f5c\u8005\u5fae\u4fe1\u53f7\uff1a<font color=\"red\">xttblog2<\/font>\u3002\u5907\u6ce8\uff1a\u201c1\u201d\uff0c\u6dfb\u52a0\u535a\u4e3b\u5fae\u4fe1\u62c9\u4f60\u8fdb\u5fae\u4fe1\u7fa4\u3002\u5907\u6ce8\u9519\u8bef\u4e0d\u4f1a\u540c\u610f\u597d\u53cb\u7533\u8bf7\u3002<\/b>\u518d\u6b21\u611f\u8c22\u60a8\u7684\u5173\u6ce8\uff01\u540e\u7eed\u6709\u7cbe\u5f69\u5185\u5bb9\u4f1a\u7b2c\u4e00\u65f6\u95f4\u53d1\u7ed9\u60a8\uff01\u539f\u521b\u6587\u7ae0\u6295\u7a3f\u8bf7\u53d1\u9001\u81f3532009913@qq.com\u90ae\u7bb1\u3002\u5546\u52a1\u5408\u4f5c\u4e5f\u53ef\u6dfb\u52a0\u4f5c\u8005\u5fae\u4fe1\u8fdb\u884c\u8054\u7cfb\uff01<\/p><p>\u672c\u6587\u539f\u6587\u51fa\u5904\uff1a<a href=\"https:\/\/www.xttblog.com\">\u4e1a\u4f59\u8349<\/a>\uff1a &raquo; <a href=\"https:\/\/www.xttblog.com\/?p=4839\">canal-adapter\u8d9f\u5751\u8bb0\u5f55<\/a><\/p><!-- <div class=\"bd-reward-stl\"><button id=\"bdRewardBtn\"><span><\/span><\/button><\/div> -->","protected":false},"excerpt":{"rendered":"<p>\u7528\u8fc7 canal \u7684\u90fd\u77e5\u9053\uff0ccanal \u73b0\u5728\u5751\u975e\u5e38\u7684\u591a\uff0c\u672c\u6587\u8bb0\u5f55\u4e00\u4e2a\u6211\u9047\u5230\u7684\u5751\u4e4b\u4e00\uff01 canal-adapte [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[1911,1938,430,1098,243],"class_list":["post-4839","post","type-post","status-publish","format-standard","hentry","category-java","tag-binglog","tag-canal","tag-430","tag-1098","tag-243"],"_links":{"self":[{"href":"https:\/\/www.xttblog.com\/index.php?rest_route=\/wp\/v2\/posts\/4839","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.xttblog.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.xttblog.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.xttblog.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.xttblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4839"}],"version-history":[{"count":1,"href":"https:\/\/www.xttblog.com\/index.php?rest_route=\/wp\/v2\/posts\/4839\/revisions"}],"predecessor-version":[{"id":4840,"href":"https:\/\/www.xttblog.com\/index.php?rest_route=\/wp\/v2\/posts\/4839\/revisions\/4840"}],"wp:attachment":[{"href":"https:\/\/www.xttblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4839"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.xttblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4839"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.xttblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4839"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}