{"id":436980,"date":"2021-04-18T06:55:22","date_gmt":"2021-04-17T22:55:22","guid":{"rendered":"http:\/\/4563.org\/?p=436980"},"modified":"2021-04-18T06:55:22","modified_gmt":"2021-04-17T22:55:22","slug":"%e7%94%a8-golang-%e7%9a%84%e5%81%9a%e4%b8%80%e4%b8%aa-pipeline-etl-%e9%97%ae%e9%a2%98%e6%9c%89%e4%bb%80%e4%b9%88%e5%a5%bd%e7%9a%84%e8%ae%be%e8%ae%a1%ef%bc%9f","status":"publish","type":"post","link":"http:\/\/4563.org\/?p=436980","title":{"rendered":"\u7528 golang \u7684\u505a\u4e00\u4e2a pipeline | ETL \u95ee\u9898\u6709\u4ec0\u4e48\u597d\u7684\u8bbe\u8ba1\uff1f"},"content":{"rendered":"<div>\n<div>\n<div>\n<h1>                  \u7528 golang \u7684\u505a\u4e00\u4e2a pipeline | ETL \u95ee\u9898\u6709\u4ec0\u4e48\u597d\u7684\u8bbe\u8ba1\uff1f               <\/h1>\n<p> <\/p>\n<div>\n<div> <span>\u8cc7\u6df1\u5927\u4f6c : yellowmarlboro <\/span>  <span><i><\/i> 1<\/span> <\/div>\n<div> <\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<div isfirst=\"1\"> <\/p>\n<pre><code>\u6709 N \u4e2a\u6570\u636e\u6e90\uff0c\u6839\u636e\u8fd9 N \u4e2a\u6570\u636e\u6e90\u8ba1\u7b97\u51fa M \u4e2a\u7ed3\u679c\uff0c\u4f9d\u8d56\u5173\u7cfb\u7b80\u5355\u5982\u4e0b\uff0c\u5b9e\u9645\u4f1a\u7565\u5fae\u590d\u6742\u4e9b\uff0c\u4f46\u4e00\u5b9a\u662f\u4e2a\u6709\u5411\u65e0\u73af\u56fe\u3002 \u7ed3\u679c\u4e5f\u6709\u6709\u53ef\u80fd\u4f1a\u5f53\u6210\u53e6\u5916\u4e00\u4e2a\u7ed3\u679c\u7684'\u6570\u636e\u6e90'      A     B    \/    \/   \/    \/  R1    R2     C             \/            \/            R3  <\/code><\/pre>\n<h5>\u6709\u4e00\u4e9b\u95ee\u9898<\/h5>\n<ul>\n<li>\u5176\u4ed6\u8c03\u7528\u65b9\u4f1a\u6765\u8c03\u7528\u6211\u670d\u52a1\uff0c\u8c03\u7528\u7684\u8f93\u5165\u4e3a\u5b57\u7b26\u4e32\u6bd4\u5982{&#8220;R2&#8221;, &#8220;R3&#8221;}\u3002<\/li>\n<li>\u6bcf\u6b21\u8981\u8ba1\u7b97\u7684\u7ed3\u679c\u7684\u6570\u91cf\u662f\u4e0d\u786e\u5b9a\u7684\uff0c\u5176\u4ed6\u8c03\u7528\u65b9\u53ef\u80fd\u53ea\u8981 R1 \u548c R2\uff0c\u90a3\u6211\u5c31\u4e0d\u60f3\u53bb\u8c03\u7528 C \u6570\u636e\u6e90\u4e86\u3002<\/li>\n<li>\u6bcf\u4e2a\u6765\u8c03\u7528\u63a5\u53e3\u7684\u72ec\u7acb\u8bf7\u6c42\u91cc\uff0c\u6211\u8fd9\u91cc\u7684\u6570\u636e\u6e90\u6211\u53ea\u60f3\u8c03\u7528\u4e00\u6b21\uff0c\u6bd4\u5982\u6211\u4e0d\u60f3\u7b97 R1,R2 \u7684\u65f6\u5019\u83b7\u53d6\u4e24\u6b21 A \u6570\u636e\u6e90\uff0c\u4ee5\u53ca\u5982\u679c\u5728\u7b97\u8fc7 R2 \u4e4b\u540e\uff0c\u518d\u53bb\u7b97 R3 \u7684\u65f6\u5019\u8981\u518d\u7b97\u4e00\u8fb9 R2 \u3002<\/li>\n<li>\u6211\u60f3\u4ece\u4e0b\u5f80\u4e0a\u7b97\uff0c\u6bd4\u5982 R3 \u7684\u903b\u8f91\u91cc\u91cc go \u51fa\u53bb R2 \u548c C\uff0cR2 \u91cc\u518d go \u51fa\u53bb A \u548c B\uff0c\u90a3\u6211\u540e\u7eed\u7b97 R1 \u7684\u65f6\u5019\uff0c\u600e\u4e48\u76f4\u63a5\u53d6\u521a\u521a\u7b97 R2 \u65f6\u7684\u83b7\u53d6\u8fc7\u7684 A\uff0cA \u5199\u6210\u5355\u4f8b\uff1f\u53ea\u8981\u88ab\u8c03\u7528\u540e\uff0c\u518d\u6b21\u88ab\u8c03\u7528\u8fd4\u56de\u6570\u636e\u6e90\uff1f -\u5f53\u7136\u8fd8\u6709\u5176\u4ed6\u7684\uff0c\u6bd4\u5982\u9519\u8bef\u4e2d\u65ad\u4e4b\u7c7b\u3002<\/li>\n<\/ul><\/div>\n<div> <b>\u5927\u4f6c\u6709\u8a71\u8aaa<\/b> (<span>2<\/span>)        <\/div>\n<div> <\/div>\n<\/p><\/div>\n<\/p><\/div>\n<ul>\n<li data-pid=\"5745177\" data-uid=\"2\">\n<div>\n<div>\n<div> <span>\u8cc7\u6df1\u5927\u4f6c : dayeye2006199 <\/span>  <\/div>\n<div> <i title=\"\u5f15\u7528\"><\/i>  <span>          <\/span> <\/div>\n<\/p><\/div>\n<div>                                                             &gt; \u5176\u4ed6\u8c03\u7528\u65b9\u4f1a\u6765\u8c03\u7528\u6211\u670d\u52a1\uff0c\u8c03\u7528\u7684\u8f93\u5165\u4e3a\u5b57\u7b26\u4e32\u6bd4\u5982{&#8220;R2&#8221;, &#8220;R3&#8221;}\u3002<br \/>&gt; \u6bcf\u6b21\u8981\u8ba1\u7b97\u7684\u7ed3\u679c\u7684\u6570\u91cf\u662f\u4e0d\u786e\u5b9a\u7684\uff0c\u5176\u4ed6\u8c03\u7528\u65b9\u53ef\u80fd\u53ea\u8981 R1 \u548c R2\uff0c\u90a3\u6211\u5c31\u4e0d\u60f3\u53bb\u8c03\u7528 C \u6570\u636e\u6e90\u4e86\u3002<\/p>\n<p>\u8fd9\u4e2a\u64cd\u4f5c\u53eb Cull\uff0c\u662f\u56fe\u4f18\u5316\u7684\u4e00\u79cd\u3002\u5177\u4f53\u505a\u6cd5\u5c31\u662f\uff0c\u5bf9\u4f60\u9700\u8981\u7684\u8f93\u51fa\uff0c\u4f8b\u5982{&#8220;R2&#8221;, &#8220;R3&#8221;}\uff0c\u83b7\u5f97\u5b83\u4eec\u7684\u6240\u6709\u4f9d\u8d56\uff0c\u548c\u4f9d\u8d56\u7684\u4f9d\u8d56&#8230; \u7136\u540e\u5c06\u56fe\u91cc\u9762\u4e0d\u5c5e\u4e8e\u4f9d\u8d56\u7684\u8282\u70b9\u5220\u9664\uff0c\u5c31\u5f97\u5230\u4e00\u4e2a\u7cbe\u7b80\u4ee5\u540e\u7684 DAG \u3002<\/p>\n<p>&gt; \u6bcf\u4e2a\u6765\u8c03\u7528\u63a5\u53e3\u7684\u72ec\u7acb\u8bf7\u6c42\u91cc\uff0c\u6211\u8fd9\u91cc\u7684\u6570\u636e\u6e90\u6211\u53ea\u60f3\u8c03\u7528\u4e00\u6b21\uff0c\u6bd4\u5982\u6211\u4e0d\u60f3\u7b97 R1,R2 \u7684\u65f6\u5019\u83b7\u53d6\u4e24\u6b21 A \u6570\u636e\u6e90\uff0c\u4ee5\u53ca\u5982\u679c\u5728\u7b97\u8fc7 R2 \u4e4b\u540e\uff0c\u518d\u53bb\u7b97 R3 \u7684\u65f6\u5019\u8981\u518d\u7b97\u4e00\u8fb9 R2 \u3002<br \/>\u6211\u60f3\u4ece\u4e0b\u5f80\u4e0a\u7b97\uff0c\u6bd4\u5982 R3 \u7684\u903b\u8f91\u91cc\u91cc go \u51fa\u53bb R2 \u548c C\uff0cR2 \u91cc\u518d go \u51fa\u53bb A \u548c B\uff0c\u90a3\u6211\u540e\u7eed\u7b97 R1 \u7684\u65f6\u5019\uff0c\u600e\u4e48\u76f4\u63a5\u53d6\u521a\u521a\u7b97 R2 \u65f6\u7684\u83b7\u53d6\u8fc7\u7684 A\uff0cA \u5199\u6210\u5355\u4f8b\uff1f\u53ea\u8981\u88ab\u8c03\u7528\u540e\uff0c\u518d\u6b21\u88ab\u8c03\u7528\u8fd4\u56de\u6570\u636e\u6e90\uff1f -\u5f53\u7136\u8fd8\u6709\u5176\u4ed6\u7684\uff0c\u6bd4\u5982\u9519\u8bef\u4e2d\u65ad\u4e4b\u7c7b\u3002<\/p>\n<p>\u8ba1\u7b97\u7684\u987a\u5e8f\u53ef\u4ee5\u5c06\u6240\u6709\u8282\u70b9\u6309\u7167\u4f9d\u8d56\u5173\u7cfb\u8fdb\u884c\u62d3\u6251\u6392\u5e8f\uff0c\u5c31\u53ef\u4ee5\u4fdd\u8bc1\u4e00\u4e2a\u8282\u70b9\u8ba1\u7b97\u7684\u65f6\u5019\uff0c\u5b83\u7684\u6240\u6709\u4f9d\u8d56\u8282\u70b9\u90fd\u662f\u88ab\u8ba1\u7b97\u8fc7\u7684\u3002<br \/>\u8ba1\u7b97\u8fc7\u7684\u8282\u70b9\u7684\u7ed3\u679c\uff0c\u53ef\u4ee5\u7528\u4e2a dict \u4fdd\u5b58\u4e00\u4e0b\uff0c\u4e0b\u6b21\u9700\u8981\u7684\u65f6\u5019\u76f4\u63a5\u6309\u7167\u8282\u70b9\u540d\u79f0\u53d6\u51fa\u5c31\u4e0d\u4f1a\u91cd\u65b0\u89e6\u53d1\u8ba1\u7b97\u4e86\u3002                                                            <\/div>\n<\/p><\/div>\n<\/li>\n<li data-pid=\"5745178\" data-uid=\"2\">\n<div>\n<div>\n<div> <span>\u4e3b<\/span> <span>\u8cc7\u6df1\u5927\u4f6c : yellowmarlboro <\/span>  <\/div>\n<div> <i title=\"\u5f15\u7528\"><\/i>  <span>          <\/span> <\/div>\n<\/p><\/div>\n<div>                                                             @dayeye2006199 \u53d7\u6559 \u975e\u5e38\u611f\u8c22~                                                            <\/div>\n<\/p><\/div>\n<\/li>\n<li>\n","protected":false},"excerpt":{"rendered":"<p>\u7528 golang \u7684\u505a\u4e00\u4e2a pip&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[],"tags":[],"_links":{"self":[{"href":"http:\/\/4563.org\/index.php?rest_route=\/wp\/v2\/posts\/436980"}],"collection":[{"href":"http:\/\/4563.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/4563.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/4563.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/4563.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=436980"}],"version-history":[{"count":0,"href":"http:\/\/4563.org\/index.php?rest_route=\/wp\/v2\/posts\/436980\/revisions"}],"wp:attachment":[{"href":"http:\/\/4563.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=436980"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/4563.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=436980"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/4563.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=436980"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}