DataX-MySQL2MongoDB案例

MySQL2MongoDB

1.Josn配置文件可能加载失败，需要核对各参数，严格按照官方样例写参数，即使是Json格式也可能解析失败！

2.源库单表数据量过大时一定要加主键，加分片键 “splitPk“，分片键必需为主键、int整数型，此键可以不包含在所传输字段中；

3.报错：

Exception in thread "taskGroup-0" com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-07], Description:[读取数据库数据失败. 请检查您的配置的 column/table/where/querySql或者向 DBA 寻求帮助.].  - 执行的SQL为: select WorkOrderId,CustStoreId,CustStoreName,CustStoreCode,PreCustStoreId,PreCustStoreName,CustSettleId,CustSettleName,IsCustomer,CustCoopType,ProCode,AppCode,ProName,CityCode,CityName,AreaCode,AreaName,InstallAddress,InstallTime,RequiredTime,LinkMan,LinkTel,SourceType,SecondLinkTel,SecondLinkMan,WarehouseId,WarehouseName,Remark,IsUrgent,CustUniqueSign,CreatePersonCode,CreatePersonName,EffectiveTime,OrderType,EffectiveSuccessfulTime,CreatedById,CreatedAt,UpdatedById,UpdatedAt,DeletedById,DeletedAt,Deleted,AppointStatus,ServiceCode,CreateType,ServiceName,RecCompleteTime,RecRecordPersonCode,RecRecordPersonName,WfOrderCompletetime,WfAssignor,WfDirectOrderAccept,WorkOdrTypeStatus,WorkOdrCloseStatusName,WorkOdrSuspendStatus,WorkStatus,WorkOdrClosedAt,AFCCreatedAt,AFCRetreatTime,AFCFollowFrequency,AFCIfRemove,AFCReturnedTypeCode,VHSAppraisalStatus,VHSReceiverLoginName,OperatingInfo,OperatedInfo,ServiceProviderCode,WorkflowInfo,WorkServiceSubject,AFCWorkFinanceRecord,AFCPaymentReport,VHSWorkFileReceiveInfo,CustomerId,CustomerName from workjsoninfo  where  (169036 <= Id AND Id < 338071)  具体错误信息为：com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Application was streaming results when the connection failed. Consider raising value of 'net_write_timeout' on the server.

解决方法：

增加源端 wait_timeout 参数值。

NULL值处理

问题：mysql2mongo数据库传输，不识别mysql内josn字段，因为datax会给json增加转义字符 “"

解决方法:源端字段类型改为varchar，入目标为array类型，mongodbwriter需要指定特殊解析数组分隔符；

注意事项

1.单 executor执行速度有限，尽量开多管道；