设为首页 - 加入收藏 PHP编程网 - PHP站长网 (http://www.52php.cn)- 电商,百科,编程,业界,移动互联,5G,云计算,站长网!
热搜: 娱乐 专业 服务 百度
当前位置: 首页 > 大数据 > 正文

数据处理---Spring Batch之实践(2)

发布时间:2021-01-24 12:23 所属栏目:[大数据] 来源:网络整理
导读:public class CustomerMapper implements FieldSetMapperCustomerCredit { @Override public CustomerCredit mapFieldSet(FieldSet fieldSet) throws BindException { CustomerCredit lv = new CustomerCredit(); l

public class CustomerMapper implements FieldSetMapper<CustomerCredit> {
@Override
public CustomerCredit mapFieldSet(FieldSet fieldSet) throws BindException {
CustomerCredit lv = new CustomerCredit();
lv.setId(Integer.parseInt(fieldSet.readString(0)));
lv.setName(fieldSet.readString(1));
lv.setCredit(fieldSet.readBigDecimal(2));
return lv;
}
}

如果是读数据库呢?就是下面那个JdbcCursorItemReader,里面指定了dataSource,sql,rowMapper,这些都类似文件

public class CustomerCreditRowMapper implements RowMapper {
? ? public static final String ID_COLUMN = "id";
? ? public static final String NAME_COLUMN = "name";
? ? public static final String CREDIT_COLUMN = "credit";

? ? public Object mapRow(ResultSet rs,int rowNum) throws SQLException {
? ? ? ? CustomerCredit customerCredit = new CustomerCredit();
? ? ? ? customerCredit.setId(rs.getInt(ID_COLUMN));
? ? ? ? customerCredit.setName(rs.getString(NAME_COLUMN));
? ? ? ? customerCredit.setCredit(rs.getBigDecimal(CREDIT_COLUMN));
? ? ? ? return customerCredit;
? ? }
}

还可以和hibernate结合来读取数据,可以读取存储过程的数据,等等,具体可以参考spring batch文档

说好的自定义处理数据呢,比如我们把每个人的Credit加10.

@Component("customProcessor")
public class CustomProcessor implements
ItemProcessor<CustomerCredit,CustomerCredit> {
@PersistenceContext
private EntityManager em;
@Override
public CustomerCredit process(CustomerCredit item) throws Exception {
System.out.println(new Date().toString()+"start to process");
if (item == null) {
return null;
}
try {
item = em.find(CustomerCredit.class,item.getId());
item.setCredit(item.getCredit().add(new BigDecimal(10)));
//find by id才可以persist (否则出现detached entity passed to persist)
em.persist(item);
} catch (Exception e) {
e.printStackTrace();
}
return item;
}
}

每一步的处理,我们想看看结果,可以定制一个ItemReadListener

public class CustomStepListener implements ItemReadListener<CustomerCredit> ? 具体代码省略

修改module-context.xml

<batch:job id="job1">
<batch:step id="step1">
<batch:tasklet transaction-manager="transactionManager"
start-limit="100">
<batch:chunk reader="itemReade" writer="itemwriter" processor="customProcessor"?
commit-interval="3" />
</batch:tasklet>
</batch:step>
</batch:job>

在META-INF下面加个persistence.xml

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="1.0"
xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd">
<persistence-unit name="spring" transaction-type="RESOURCE_LOCAL">
<!--?
<class>com.test.jpatest.model.Customer</class>
<class>com.test.jpatest.model.Address</class>
-->
<provider>org.hibernate.ejb.HibernatePersistence</provider>
</persistence-unit>
</persistence>

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'exampleConfiguration': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.batch.core.repository.JobRepository com.test.batch.ExampleConfiguration.jobRepository; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jobRepository': Cannot resolve reference to bean 'transactionManager' while setting bean property 'transactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager' defined in class path resource [launch-context.xml]: Cannot resolve reference to bean 'entityManagerFactory' while setting bean property 'entityManagerFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [launch-context.xml]: Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: No persistence unit with name 'spring' found

运行的时候遇到上面的错误,就是没有上面的配置的原因

org.springframework.transaction.InvalidIsolationLevelException: Standard JPA does not support custom isolation levels - use a special JpaDialect for your JPA implementation
at org.springframework.orm.jpa.DefaultJpaDialect.beginTransaction(DefaultJpaDialect.java:66)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.beginTransaction(HibernateJpaDialect.java:59)

遇到这样的错误,是因为默认的JPA不支持自定义的事物隔离级别。可以自定义一个CustomHibernateJpaDialect extends HibernateJpaDialect,具体代码没有列出,可以找下。


在用Quartz的时候遇到

Caused by: java.lang.IncompatibleClassChangeError: class org.springframework.scheduling.quartz.CronTriggerBean has interface org.quartz.CronTrigger as super class ?

这个是由于网上的很多例子都是quartz版本稍旧的原因,我用的是quartz?2.1.7


错误:Jobs added with no trigger must be durable
<property name="durability" value="true" /> ?

坑真的是不少,需要一个个解决。最后测试一下,是不是定时执行我们的job:

public class App {
public static void main(String[] args) {
String springConfig = "launch-context.xml";
ApplicationContext context = new ClassPathXmlApplicationContext(
springConfig);
}
}

祝你好运,能够成功,

数据处理---Spring Batch之实践

【免责声明】本站内容转载自互联网,其相关言论仅代表作者个人观点绝非权威,不代表本站立场。如您发现内容存在版权问题,请提交相关链接至邮箱:bqsm@foxmail.com,我们将及时予以处理。

推荐文章
热点阅读