관심분야/모바일서버

제주버스OpenApi 적용방식(2)

을량 2019. 7. 16. 14:27

3-3) 배치 개발  
- 버스관련 대용량 정보를 OpenApi를 호출하여 변경처리후 적재하여 최신의 정보를 유지한다.

- 중국어 버전 대응을 위해 구글클라우드 플랫폼 TTS API 연동 
  - OpenApi로 제공되는 번역기중 테스트 결과는 구글 TTS Api가 가장 뛰어나 선정했음 
  - 구글클라우드 플랫폼 회원가입 후 결제수단으로 카드를 등록해서 사용 기본이 $300 서비스로 제공함
  - CREATE CREDENTIALS -> 엑세스키 생성하여 인증키를 해당 서버에 등록처리하여야 사용가능함

 

배치개발 방식

Spring Boot Batch 로 개발 하였다 

총 6개의 테이블을 처리한다. 

1) 전체 노선 리스트 정보(http://busopen.jeju.go.kr/OpenAPI/service/bis/Bus) 처리

    - 한국어, 중국어(구글TTS이용) 처리 

2) 노선 상세정보 (노선에 해당되는 정류장 리스트)(http://busopen.jeju.go.kr/OpenAPI/service/bis/BusLocation) 처리

3) 전체 정류장 리스트 정보(http://busopen.jeju.go.kr/OpenAPI/service/bis/Station) 처리 

    - 한국어, 중국어(구글TTS이용) 처리 

4) 정류장 경유 노선 리스트 정보(http://busopen.jeju.go.kr/OpenAPI/service/bis/StationRoutePs) 처리

5) 전체 버스 리스트 정보(http://busopen.jeju.go.kr/OpenAPI/service/bis/BusVehicle) 처리

6) 버스운행시간표정보(https://bus.jeju.go.kr/data/schedule/getScheduleByLineNum) 처리

 

메인호출 메소드

	/**
	 * 배치 Job
	 * 
	 * baseStep 호출한다
	 * 
	 * @return
	 */
	@Bean
	@Scope("prototype")
	public Job baseJob() {
		// .next 로 추가 스텝을 호출할수 있음
		return this.jobBuilderFactory.get("[Job - " + "BusIfUpdateDailyBat" + "]")
				.start(stepOne1()).next(stepTwo1()).next(stepThree1())
				.next(stepOne2()).next(stepTwo2()).next(stepThree2())
				.next(stepOne3()).next(stepTwo3()).next(stepThree3())
				.next(stepOne4()).next(stepTwo4()).next(stepThree4())
				.next(stepOne5()).next(stepTwo5()).next(stepThree5())
				.next(stepOne6()).next(stepTwo6()).next(stepThree6())
					.build();
	}

1) 전체 노선 리스트 정보(http://busopen.jeju.go.kr/OpenAPI/service/bis/Bus) 처리 소스

	/**
	 * 배치 Step1
	 */
	@Bean
	@Scope("prototype")
	public Step stepOne1() { // chunk 큰덩어리 프로세스단위
		// chunk 큰덩어리 프로세스단위 트랜잭션 단위로 생각하면됨
		Step step = this.stepBuilderFactory.get("[Step1 - " + "BusIfUpdateDailyBat1" + "]")
				.<String, String>chunk(1)
 				.reader(createRouMTableReader())
 				.processor(notProcessor())
				.build();
		return step;
	}
    
    public ItemReader<String> createRouMTableReader() {

	ItemReader<String> reader = new ItemReader<String>() {
	
		@Override
		public String read()
				throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
			
			usrTemplate = getJdbcTemplate();

			String DROP_TABLE1 = "DROP TABLE IF EXISTS T_BUS_ROU_M_NEW";
			batLogger.info("T_BUS_ROU_M_NEW DROP_TABLE : " + DROP_TABLE1);
			usrTemplate.execute(DROP_TABLE1);
			
			usrTemplate = getJdbcTemplate();
			
			String DROP_TABLE2 = "DROP TABLE IF EXISTS T_BUS_ROU_M_OLD";
			batLogger.info("T_BUS_ROU_M_OLD DROP_TABLE : " + DROP_TABLE2);
			usrTemplate.execute(DROP_TABLE2);
			
			usrTemplate = getJdbcTemplate();
			String CREATE_TABLE = "CREATE TABLE `T_BUS_ROU_M_NEW` (\r\n" + 
					"	`ROU_ID` CHAR(9) NOT NULL COMMENT '노선ID' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`LAG_CCD` CHAR(2) NOT NULL DEFAULT '01' COMMENT '언어구분코드' COLLATE 'utf8mb4_unicode_ci', \r\n" + 
					"	`SYS_ROU_NM` VARCHAR(20) NULL DEFAULT NULL COMMENT '시스템노선명' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`ROU_NM` VARCHAR(200) NULL DEFAULT NULL COMMENT '노선명' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`BUS_N` VARCHAR(20) NULL DEFAULT NULL COMMENT '버스번호' COLLATE 'utf8mb4_unicode_ci',\r\n" + 
					"	`SR_BTP_N` CHAR(9) NULL DEFAULT NULL COMMENT '시작정류장번호' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`ED_BTP_N` CHAR(9) NULL DEFAULT NULL COMMENT '종료정류장번호' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`DSL_BTP_CNT` INT(5) NULL DEFAULT NULL COMMENT '경유정류장수',\r\n" + 
					"	`ROU_RUN_DSA` INT(10) NULL DEFAULT NULL COMMENT '노선운행거리',\r\n" + 
					"	`ROU_ALT_DT` VARCHAR(30) NULL DEFAULT NULL COMMENT '노선업데이트일시' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`EL_F` CHAR(1) NOT NULL DEFAULT 'N' COMMENT '삭제여부' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`RG_PPLE` VARCHAR(20) NULL DEFAULT NULL COMMENT '등록자' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`RG_DT` TIMESTAMP NULL DEFAULT CURRENT_TIMESTAMP COMMENT '등록일시',\r\n" + 
					"	`ALT_PPLE` VARCHAR(20) NULL DEFAULT NULL COMMENT '수정자' COLLATE 'UTF8MB4_UNICODE_CI',\r\n" + 
					"	`ALT_DT` TIMESTAMP NULL DEFAULT '0000-00-00 00:00:00' COMMENT '수정일시',\r\n" + 
					"	PRIMARY KEY (`ROU_ID`, `LAG_CCD`)\r\n" + 
					")\r\n" + 
					"COMMENT='버스노선기본(전체 노선 리스트 정보)'\r\n" + 
					"COLLATE='UTF8MB4_UNICODE_CI'\r\n" + 
					"ENGINE=INNODB ";
			batLogger.info("Started CREATE_TABLE : " + CREATE_TABLE);
			usrTemplate.execute(CREATE_TABLE);			
			
			return null;
		}
	};
	/**
	 * 배치 Step2
	 */
	@Bean
	@Scope("prototype")
	public Step stepTwo1() { 
		// chunk 큰덩어리 프로세스단위 트랜잭션 단위로 생각하면됨
		Step step = this.stepBuilderFactory.get("[Step2 - " + "BusIfUpdateDailyBat1" + "]")
				.<Map<Integer, Object>, Map<Integer, Object>>chunk(1)
				.reader(itemRouMReader())
 				.processor(itemRouMProcessor())
 				.build();
		return step;
	}
    
    @Bean
public ItemProcessor<Map<Integer, Object>, Map<Integer, Object>> itemRouMProcessor() {
	
	return new ItemProcessor<Map<Integer, Object>, Map<Integer, Object>>() 
	{	
		@Override
		public Map<Integer, Object> process(Map<Integer, Object> item) throws Exception {
	
			String urlStr = "http://busopen.jeju.go.kr/OpenAPI/service/bis/Bus?serviceKey=인증키";
			
			List<String> list = new ArrayList<String>();
			list.add("dstStationId");
			list.add("orgtStationId");
			list.add("routeId");
			list.add("routeLen");
			list.add("routeNm");
			list.add("routeNum");
			list.add("routeSubNm");
			list.add("stationCnt");
			list.add("upd");
			list.add("useYn");
			List<Map<String,String>> rtnlist = BusXmlUtil.getXmlIfBySBox(urlStr, "item", list);
			usrTemplate = getJdbcTemplate();
			Translate translate = TranslateOptions.newBuilder().build().getService();
			String SAVE_JOB_EXECUTION ="";
			StringBuilder sb = new StringBuilder();
			for (int i = 0; i < rtnlist.size(); i++) {
				Map<String,String> dataMap =(Map<String,String>)rtnlist.get(i);
				String elF ="N".equals((String)dataMap.get("useYn"))?"Y":"N";
				
				if(((String)dataMap.get("routeId")).startsWith("407")||((String)dataMap.get("routeId")).startsWith("408"))
				{
					continue;
				}
				
				String TRX_TT="";
				try {
					Translation translation2 =
					        translate.translate(
					        		dataMap.get("routeSubNm"),
					            TranslateOption.sourceLanguage("ko"),
					            TranslateOption.targetLanguage("zh-CN"));
					TRX_TT = translation2.getTranslatedText();
				} catch (Exception e) {
					e.printStackTrace();
				}
				sb.setLength(0);
				sb.append(TRX_TT).append(" [").append(dataMap.get("routeSubNm")).append("]");

				SAVE_JOB_EXECUTION = " INSERT INTO  T_BUS_ROU_M_NEW ( ROU_ID, LAG_CCD, SYS_ROU_NM, ROU_NM, BUS_N, SR_BTP_N, ED_BTP_N, DSL_BTP_CNT, ROU_RUN_DSA, ROU_ALT_DT, EL_F, RG_PPLE, ALT_PPLE ) " + 
				"VALUES('"+dataMap.get("routeId")+"', '01','"+dataMap.get("routeNm")+"', '"+dataMap.get("routeSubNm")+"', '"+dataMap.get("routeNum")+"', '"+dataMap.get("orgtStationId")+"', '"
						+dataMap.get("dstStationId")+"', '"+dataMap.get("stationCnt")+"', '"+dataMap.get("routeLen")+"', '"+dataMap.get("upd")+"', '"+elF+"', '0000000001', '0000000001' )   ";
				usrTemplate.execute(SAVE_JOB_EXECUTION);
				
				SAVE_JOB_EXECUTION = " INSERT INTO  T_BUS_ROU_M_NEW ( ROU_ID, LAG_CCD, SYS_ROU_NM, ROU_NM, BUS_N, SR_BTP_N, ED_BTP_N, DSL_BTP_CNT, ROU_RUN_DSA, ROU_ALT_DT, EL_F, RG_PPLE, ALT_PPLE ) " + 
				"VALUES('"+dataMap.get("routeId")+"', '02','"+dataMap.get("routeNm")+"', '"+new String(sb.toString().getBytes(),"utf-8")+"', '"+dataMap.get("routeNum")+"', '"+dataMap.get("orgtStationId")+"', '"
						+dataMap.get("dstStationId")+"', '"+dataMap.get("stationCnt")+"', '"+dataMap.get("routeLen")+"', '"+dataMap.get("upd")+"', '"+elF+"', '0000000001', '0000000001' )   ";
				usrTemplate.execute(SAVE_JOB_EXECUTION);
			}
			
			Map<Integer, Object> map = new HashMap<Integer, Object>();
			return map;
		}
	};
}
	/**
	 * 배치 Step3
	 */
	@Bean
	@Scope("prototype")
	public Step stepThree1() { // chunk 큰덩어리 프로세스단위
		// chunk 큰덩어리 프로세스단위 트랜잭션 단위로 생각하면됨
		Step step = this.stepBuilderFactory.get("[Step3 - " + "BusIfUpdateDailyBat1" + "]")
				.<String, String>chunk(1)
 				.reader(reNameRouMTableReader())
 				.processor(notProcessor())
				.build();
		return step;
	}
    
    public ItemReader<String> reNameRouMTableReader() {

	ItemReader<String> reader = new ItemReader<String>() {
	
		@Override
		public String read()
				throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
			
			usrTemplate = getJdbcTemplate();
		
			String RENAME_TABLE1 = "RENAME TABLE T_BUS_ROU_M TO T_BUS_ROU_M_OLD ";
			batLogger.info("Started RENAME_TABLE1 : " + RENAME_TABLE1);
			usrTemplate.execute(RENAME_TABLE1);
			
			String RENAME_TABLE2 = "RENAME TABLE T_BUS_ROU_M_NEW TO T_BUS_ROU_M ";
			batLogger.info("Started RENAME_TABLE2 : " + RENAME_TABLE2);
			usrTemplate.execute(RENAME_TABLE2);
		
			return null;
		}
	};

	return reader;
}

 

'관심분야 > 모바일서버' 카테고리의 다른 글

CORS 해결하기(안드로메다 갔다옴)  (0) 2019.11.05
AWS MySQL 설치된 EC2 인스턴스 HeidSQL 연동 절차  (0) 2019.09.11
Spring 3.x F/W 기본 구조  (0) 2019.08.21
제주버스OpenApi 적용방식(1)  (0) 2019.07.16

'관심분야/모바일서버'의 다른글

  • 현재글제주버스OpenApi 적용방식(2)

관련글

티스토리툴바