This topic describes how to migrate applications from the High-Speed Service Framework (HSF) to the Apache Dubbo (Dubbo) framework by using Ali-Tomcat.
Migration solutions
The final goal of migration is to migrate applications from HSF and the registry of Serverless App Engine (SAE) to Dubbo and the Nacos registry. Two solutions are available:
- Two-step migration
- Migrate from HSF and the registry of SAE to Dubbo and the registry of SAE.
- Migrate from the registry of SAE to the Nacos registry.
This solution features high stability and is suitable for small-step iteration. However, the application must be published twice.
- Direct migration
Migrate from HSF and the registry of SAE to Dubbo and the Nacos registry.
HSF does not support Nacos. Additional development is required.
If you want to quickly migrate your application to Dubbo and release it, we recommend that you use the first solution for stability consideration. The following section describes how to perform a two-step migration.
Migration architecture
A Dubbo service is registered in both HSF and Dubbo formats to ensure that the HSF service consumer discovers the Dubbo service. The Dubbo service consumer subscribes to the data in the HSF and Dubbo formats to ensure that the Dubbo service consumer discovers the HSF service.
Prerequisites
The following components required for migration are deployed:
- Start the lightweight configuration center
- Dubbo 2.7.3
- EDAS-container V3.5.5
- edas-dubbo-extension 2.0.6
├── pom.xml
├── src
│ └── main
│ └── java
│ └── com
│ └── alibaba
│ └── edas
│ └── DemoService.java
Migrate the service provider
Assume that you want to migrate the HSF application edas-hsf-demo-provider-war, which contains the following files:
pom.xml
is the configuration file for dependencies between modules of the application.DemoServiceImpl.java
is the implementation of DemoService.hsf-provider-beans.xml
is the Spring bean declaration file of HSF.web.xml
is the descriptor used for WAR package deployment.
edas-hsf-demo-provider-war
directory: ├── pom.xml
├── src
│ ├── main
│ │ ├── java
│ │ │ └── com
│ │ │ └── alibaba
│ │ │ └── edas
│ │ │ └── hsf
│ │ │ └── provider
│ │ │ └── DemoServiceImpl.java
│ │ ├── resources
│ │ │ └── hsf-provider-beans.xml
│ │ └── webapp
│ │ └── WEB-INF
│ │ └── web.xml
- Add Dubbo dependencies to pom.xml.We recommend that you use Spring 4.x or later for HSF services.
- Change hsf-provider-beans.xml to dubbo-provider-beans.xml.
The following information shows the configuration of the hsf-provider-beans.xml file:
<? xml version="1.0" encoding="UTF-8"? > <beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:hsf="http://www.taobao.com/hsf" xmlns="http://www.springframework.org/schema/beans" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.taobao.com/hsf http://www.taobao.com/hsf/hsf.xsd" default-autowire="byName"> <bean id="itemService" class="com.alibaba.edas.hsf.provider.DemoServiceImpl" /> <! -- Example of providing a service --> <hsf:provider id="demoService" interface="com.alibaba.edas.DemoService" ref="itemService" version="1.0.0"> </hsf:provider> </beans>
You must change it to
dubbo-provider-beans.xml
:<? xml version="1.0" encoding="UTF-8"? > <beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dubbo="http://dubbo.apache.org/schema/dubbo" xmlns="http://www.springframework.org/schema/beans" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.3.xsd http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd"> <dubbo:application name="edas-dubbo-demo-provider"/> <dubbo:registry id="edas" address="edas://127.0.0.1:8080"> <! -- This means Dubbo services will be registered as HSF format, so that hsf consumer can discover it. --> <dubbo:parameter key="hsf.enable" value="true"/> </dubbo:registry> <bean id="demoService" class="com.alibaba.edas.dubbo.provider.DemoServiceImpl"/> <dubbo:service interface="com.alibaba.edas.DemoService" ref="demoService" group="HSF" version="1.0.0"/> </beans>
Note- The address of the Dubbo registry must be set to
edas://127.0.0.1:8080
. It must start with edas. The IP address and port number can remain unchanged, which will be automatically replaced with those of the online service by SAE during deployment. - You must add
<dubbo:parameter key="hsf.enable" value="true"/>
, which means that the Dubbo service is registered in both HSF and Dubbo formats to ensure that the HSF client discovers the service. - You must specify the group and version for the
<dubbo:service>
label. The default group isHSF
and the default version is1.0.0
. Otherwise, the HSF client cannot be called.
- The address of the Dubbo registry must be set to
- In the web.xml file, replace hsf-provider-beans.xml with dubbo-provider-beans.xml.
You need only to replace
hsf-provider-beans.xml
withdubbo-provider-beans.xml
.<! DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc. //DTD Web Application 2.3//EN" "http://java.sun.com/dtd/web-app_2_3.dtd" > <web-app> <display-name>Archetype Created Web Application</display-name> <context-param> <param-name>contextConfigLocation</param-name> <param-value>classpath:dubbo-provider-beans.xml</param-value> </context-param> <listener> <listener-class> org.springframework.web.context.ContextLoaderListener </listener-class> </listener> </web-app>
- Verify the result in a local environment.Verification in a local environment includes verifying whether the service can be registered and verifying whether the service consumer can call the service.
Migrate the service consumer
Based on edas-hsf-demo-consumer-war, migrate the service consumer to edas-dubbo-demo-consumer-war.
- Add Dubbo dependencies to pom.xml.The service consumer can be migrated in the same way as the service provider. Main operations include adding dubbo and dubbo-edas-extension dependencies and deleting the edas-sdk dependency. For more information, see Add Dubbo dependencies to pom.xml in the "Migrate the service provider" section.
- Change hsf-comsumer-beans.xml to dubbo-consumer-beans.xml.The following information shows the configuration of the hsf-consumer-beans.xml file:
<? xml version="1.0" encoding="UTF-8"? > <beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:hsf="http://www.taobao.com/hsf" xmlns="http://www.springframework.org/schema/beans" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.taobao.com/hsf http://www.taobao.com/hsf/hsf.xsd" default-autowire="byName"> <! -- Example of consuming a service --> <hsf:consumer id="demoService" interface="com.alibaba.edas.DemoService" version="1.0.0"> </hsf:consumer> </beans>
Change the file to dubbo-consumer-beans.xml.
<? xml version="1.0" encoding="UTF-8"? > <beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dubbo="http://dubbo.apache.org/schema/dubbo" xmlns="http://www.springframework.org/schema/beans" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.3.xsd http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd"> <dubbo:application name="edas-dubbo-demo-consumer"/> <dubbo:registry id="edas" address="edas://127.0.0.1:8080"> <! -- This means Dubbo consumer will subscribe HSF services --> <dubbo:parameter key="hsf.enable" value="true"/> </dubbo:registry> <dubbo:reference id="demoService" interface="com.alibaba.edas.DemoService" group="HSF" version="1.0.0" check="false"/> </beans>
Note- The address of the Dubbo registry must start with
edas://
. - Specify
group
andversion
for the<dubbo:service>
label. Make sure thatgroup
andversion
are the same as those of the service provider. The defaultgroup
is HSF and the defaultversion
is 1.0.0. - Add the
check="false"
configuration, which means that the service consumer application will not immediately fail without the service provider address when the application attempts to start. - Add the configuration
<dubbo:parameter key="hsf.enable" value="true"/>
, which means that the service consumer subscribes to the data of the service provider.
- The address of the Dubbo registry must start with
- In the web.xml file, replace hsf-consumer-beans.xml with dubbo-comsumer-beans.xml.
- Verify the result in a local environment.Verification in a local environment includes verifying whether the service is registered with the light-weight configuration registry and verifying whether the HSF and Dubbo services can be called.
Deploy applications to SAE and verify the result
In SAE, create the following four applications:
- edas-dubbo-demo-consumer is the service consumer application after migration. The runtime environment is Apache Tomcat 7.0.91.
- edas-dubbo-demo-provider is the service provider application after migration. The runtime environment is Apache Tomcat 7.0.91.
- edas-hsf-demo-consumer is the service consumer application before migration. The runtime environment is EDAS-Container V3.5.4.
- edas-hsf-demo-provider is the service provider application before migration. The runtime environment is EDAS-Container V3.5.4.
- Deploy the four WAR packages to the four applications separately. For more information, see Deploy applications to SAE.
- Run the following command to check whether the service consumer edas-hsf-demo-consumer can call the HSF and Dubbo service providers:
curl http://39.106.XX.XXX:8080/index.htm
- View logs in the standard output of Ali-Tomcat of the edas-hsf-demo-consumer application, such as /home/admin/taobao-tomcat-production-7.0.XX.X/logs/catalina.out.
If the output is similar to the following information, the HSF service consumer has consumed the HSF and Dubbo services.
- Run the following command to check whether edas-dubbo-demo-consumer can call the HSF and Dubbo service providers.
curl http://192.168.XX.XX:8080/index.htm
- View logs in the standard output of Apache Tomcat of the edas-dubbo-demo-consumer application, such as /home/admin/apache-tomcat-7.0.91/logs/catalina.out.
If the output is similar to the following information, the Dubbo service consumer has consumed the HSF and Dubbo services.
FAQ
- Why am I unable to find the service provider address after the Dubbo service consumer is started?
Problem description
java.lang.IllegalStateException: Failed to check the status of the service com.xxxx.xxxxx.service.xxxxxConfigService. No provider available for the service HSF/com.xxxxx.xxxxx.service.xxxxxxxxxxService:1.0.0 from the url edas://127.0.0.1:8080/org.apache.dubbo.registry.RegistryService? application=xxxx-flow-center-bj&dubbo=2.0.2&group=HSF&interface=com.xxxx.xxxxxx.service.xxxxxxxxxxService&lazy=false&methods=queryConfigs,getConfig,saveConfig&pid=11596®ister.ip=xxx.xx.xx.xxx&release=2.7.3&revision=1.0.1-SNAPSHOT&side=consumer&sticky=false&timeout=2000×tamp=1564242421194&version=1.0.0 to the consumer xxx.xx.xx.xxx use dubbo version 2.7.3
Possible cause
The registry asynchronously pushes service provider addresses. By default, Dubbo checks whether a service provider address is available during startup. If no address is available, this error occurs.
Solution
In the
<dubbo:reference>
label of Dubbo, add the followingcheck="false"
configuration:<dubbo:reference id="demoService" interface="com.alibaba.edas.DemoService" group="HSF" version="1.0.0" check="false"/>
After the configuration is added, Dubbo does not check whether the service provider address is available during startup. However, if the Dubbo service needs to be called in business initialization code, the business may fail.
- Why does an error occur when the HSF service consumer calls the Dubbo service?
Problem description
2019-07-28 23:07:38.005 [WARN ] [cf67433d1e7a44412a518bd190100d176-node401] [NettyServerWorker-6-1] [o.a.d.r.exchange.codec.ExchangeCodec:91] | [DUBBO] Fail to encode response: Response [id=343493, version=HSF2.0, status=20, event=false, error=null, result=AppResponse [value=FlowControlDto(postWeightDtoHashMap={614215325=PostWeightDto(postId=614215325, weight=1.0, postSourceType=null)}), exception=null]], send bad_response info instead, cause: For input string: "", dubbo version: 2.7.3, current host: xxx.xx.xx.xxx java.lang.NumberFormatException: For input string: "" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:592) at java.lang.Integer.parseInt(Integer.java:615) at org.apache.dubbo.common.Version.parseInt(Version.java:133) at org.apache.dubbo.common.Version.getIntVersion(Version.java:118) at org.apache.dubbo.common.Version.isSupportResponseAttachment(Version.java:102) at org.apache.dubbo.rpc.protocol.dubbo.DubboCodec.encodeResponseData(DubboCodec.java:195) at org.apache.dubbo.remoting.exchange.codec.ExchangeCodec.encodeResponse(ExchangeCodec.java:283) at org.apache.dubbo.remoting.exchange.codec.ExchangeCodec.encode(ExchangeCodec.java:71) at org.apache.dubbo.rpc.protocol.dubbo.DubboCountCodec.encode(DubboCountCodec.java:40) at org.apache.dubbo.remoting.transport.netty4.NettyCodecAdapter$InternalEncoder.encode(NettyCodecAdapter.java:70) ...
Possible cause
After Dubbo is upgraded to V2.7, HSF is not compatible with Dubbo.
Solution
Upgrade EDAS-Container to V3.5.5. This issue is fixed in EDAS-Container of this version in HSF.
- Why does the Dubbo service consumer fail to call the HSF service provider?
Problem description
java.lang.Exception: [HSF-Provider-192.168.0.46] Error log: [HSF-Provider] App [xxxxxxx-3b6f-42d3-xxxx-0ad2434xxxxx] failed to verify the caller signature [null] for [com.alibaba.edas.DemoService:1.0.0] [sayHello] from client [192.168.XX.XX] com.taobao.hsf.io.remoting.dubbo2.Dubbo2PacketFactory.serverCreate(Dubbo2PacketFactory.java:284) com.taobao.hsf.io.stream.AbstractServerStream.write(AbstractServerStream.java:25) com.taobao.hsf.io.RpcOutput.flush(RpcOutput.java:37) com.taobao.hsf.remoting.provider.ProviderProcessor$OutputCallback.operationComplete(ProviderProcessor.java:155) com.taobao.hsf.remoting.provider.ProviderProcessor$OutputCallback.operationComplete(ProviderProcessor.java:130) com.taobao.hsf.util.concurrent.AbstractListener.run(AbstractListener.java:18) com.taobao.hsf.invocation.AbstractContextAwareRPCCallback.access$001(AbstractContextAwareRPCCallback.java:12) com.taobao.hsf.invocation.AbstractContextAwareRPCCallback$1.run(AbstractContextAwareRPCCallback.java:27) com.taobao.hsf.util.concurrent.WrappedListener.run(WrappedListener.java:34) com.taobao.hsf.invocation.AbstractContextAwareRPCCallback.run(AbstractContextAwareRPCCallback.java:36) com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:595) com.taobao.hsf.util.concurrent.DefaultListenableFuture.addListener(DefaultListenableFuture.java:32) com.taobao.hsf.remoting.provider.ProviderProcessor.handleRequest(ProviderProcessor.java:55) com.taobao.hsf.io.remoting.dubbo2.message.Dubbo2ServerHandler$1.run(Dubbo2ServerHandler.java:65) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748)
Possible cause
Call authentication is enabled for HSF, but Dubbo does not support this authentication.
Solution
Add
-DneedAuth=false
to the HSF service provider to disable call authentication. - Why does the Dubbo service consumer fail to call the HSF service provider?
Problem description
2019-08-02 17:17:15.187 [WARN ] [cf67433d1e7a44412a518bd190100d176-node401] [NettyClientWorker-4-1] [o.a.d.r.p.dubbo.DecodeableRpcResult:91] | [DUBBO] Decode rpc result failed: null, dubbo version: 2.7.3, current host: xxx.xx.xx.xxx java.lang.StackOverflowError: null at sun.reflect.UnsafeFieldAccessorImpl.ensureObj(UnsafeFieldAccessorImpl.java:57) at sun.reflect.UnsafeByteFieldAccessorImpl.setByte(UnsafeByteFieldAccessorImpl.java:98) at java.lang.reflect.Field.setByte(Field.java:838) at com.alibaba.com.xxxxxx.hessian.io.JavaDeserializer$ByteFieldDeserializer.deserialize(JavaDeserializer.java:452) at com.alibaba.com.xxxxxx.hessian.io.JavaDeserializer.readObject(JavaDeserializer.java:276) at com.alibaba.com.xxxxxx.hessian.io.JavaDeserializer.readObject(JavaDeserializer.java:203) at com.alibaba.com.xxxxxx.hessian.io.SerializerFactory.readObject(SerializerFactory.java:532) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObjectInstance(Hessian2Input.java:2820) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2743) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2278) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2080) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2074) at com.alibaba.com.xxxxxx.hessian.io.JavaDeserializer$ObjectFieldDeserializer.deserialize(JavaDeserializer.java:406) at com.alibaba.com.xxxxxx.hessian.io.JavaDeserializer.readObject(JavaDeserializer.java:276) at com.alibaba.com.xxxxxx.hessian.io.JavaDeserializer.readObject(JavaDeserializer.java:203) at com.alibaba.com.xxxxxx.hessian.io.SerializerFactory.readObject(SerializerFactory.java:532) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObjectInstance(Hessian2Input.java:2820) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2743) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2278) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2080) at com.alibaba.com.xxxxxx.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2074) at com.alibaba.com.xxxxxx.hessian.io.JavaDeserializer$ObjectFieldDeserializer.deserialize(JavaDeserializer.java:406) ...
Possible cause
The HSF service provides and depends on an earlier Hessian Lite version, which does not support serialization of JDK 8 LocalDateTime.
Solution
Upgrade EDAS-Container of the HSF service provider to V3.5.5.