作者存档: 朱坤乾

使用ndk gcc编译

半年用一次,每次用都得现查一遍。

这里记录一下ndk中使用gcc编译直接跑在android上的二进制程序

先直接生成GNU Android Toolchain,然后就可以直接使用gcc来编译了,不必使用ndk-build,也免去了一堆配置文件的要求。

$NDK/build/tools/make-standalone-toolchain.sh --toolchain=arm-linux-androideabi-4.8 --platform=android-19 --install-dir=/opt/ndk_toolchain

生成后直接设置是否可用:

echo “main(){}” | /opt/ndk_toolchain/bin/arm-linux-androideabi-gcc -x c –

没有报错,即toolchain环境已经配置好。

 

生成的文件在虚拟器中测试时,注意各家的adb文件互相冲突。比如pp助手的adb就会占用端口,需要提前查看下是否存在后台进程,存再则杀掉。

夜神则直接使用nox_adb来连接。

 

kotlin工程

最早建过一个kotlin工程,尝试了下kotlin的语法。

今天准备为我的app做一个收集日志的service时,发现eclipse中kotlin环境怎么配都无法成功。这个工程是在springboot上建立bootstrap下载下来后使用的。

仔细对比了两个工程,发现,我之前建立的工程中有kotlin builder,而新的工程中则是gradle builder。

找到原因了,在工程名上右键 configure kotlin->add native plugin,然后会添加上kotlin builder。

但是看网上信息,native plugin只是解决java和kotlin代码同时在工程中的问题。

在eclipse中没找到办法解决,在idea中尝试一下。

kotlin中报无法支持1.8的lib,在idea中可以修改kotlin支持1.8,但eclipse中未找到哪里可以修改。

build.gradle中各种插件问题修复后,在idea中可以正常使用了。现在就回到最初的问题了,在idea中运行时,怎么优先使用test/resources。

最终也没好办法,写了一个testcase来启动springboot。但在testcase中启动后就立即退出是怎么回事?查找下原因。立即即出,就在启动springboot后,sleep(xxxxxx)的时间来阻止进程进出就可以了。

 

目前的几种解决方案:

1、写一个testcase启动springboot,直接会使用test resources。(成功)

2、直接启动App.java或使用gradle bootrun,通过 vm parameter将环境变量传递过去。(正在测试中)

3、写一个properties文件,放在test resources下,启动时,优先去读取properties中设置的属性。(或者放在正式目录下的resources下)

 

关于如何传递环境变量(程序可以直接读取环境变量,因此可以不使用-D来传递)

1、jar包传递环境变量

gradle bootjar打包出来的jar包,可以用java -Dlogback.syslogHost=xxx -jar xxx.jar传递环境变量。

2、gradle bootRun 使用-D传递环境变量,需要设置以下属性。

bootRun {
    //classpath configurations.testRuntime
   systemProperties = System.properties
}

3、再查看下怎么使用bootrun从properties中设置环境变量。

在junit中,会加载logback-test.properties,那能在debug模式中使用吗。只要是junit都会加载test resources。

那这样来说,debug模式本就不应该去加载test resources,也是正确的处理方式,只有test模式时才需要去加载 test resources。

费这牛鼻子劲干啥,直接写个脚本,参数通过环境变量传递不就OK啦。

或者bootrun时,能判断出来,主动去加载一个也可以。

spring boot 加载顺序

https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html

Spring Boot uses a very particular PropertySource order that is designed to allow sensible overriding of values. Properties are considered in the following order:

    Devtools global settings properties on your home directory (~/.spring-boot-devtools.properties when devtools is active).
    @TestPropertySource annotations on your tests.
    @SpringBootTest#properties annotation attribute on your tests.
    Command line arguments.
    Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property).
    ServletConfig init parameters.
    ServletContext init parameters.
    JNDI attributes from java:comp/env.
    Java System properties (System.getProperties()).
    OS environment variables.
    A RandomValuePropertySource that has properties only in random.*.
    Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).
    Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).
    Application properties outside of your packaged jar (application.properties and YAML variants).
    Application properties packaged inside your jar (application.properties and YAML variants).
    @PropertySource annotations on your @Configuration classes.
    Default properties (specified by setting SpringApplication.setDefaultProperties).

还得看下spring loaded与kotlin的结合中。

还得看下devtool的作用。

下面开始测试下 devtool 与 springloaded,主要就是controller,接口增加,修改,删除是否能立即生效。

记得之前用eclipse+java app+springloaded,增加方法都支持。

在外部运行gradlew.bat bootrun

1、使用devtool

2、使用springloaded,从环境变量传入

3、使用springloaded,从build.gradle中配置

直接运行App.kt

1、好像只能使用环境变量来配置了吧。

unity install location最好设置为Automatic

Unity install location,根据 unity 官方文档:https://docs.unity3d.com/Manual/class-PlayerSettingsAndroid.html

Install Location 	Specifies application install location on the device (for detailed information, refer to Android Developer documentation on install locations.
        Automatic 	Let the operating system decide. User will be able to move the app back and forth.
        Prefer External 	Install the application to external storage (SD card) if possible. The operating system does not guarantee it; if not possible, the app will be installed to internal memory.
        Force Internal 	Force the application to be installed to internal memory. The user will be unable to move the app to external storage.

项目组最初选择的是 Prefer External,本意是想尽量安装在SD卡上,不占用玩家手机内存储。

今天在一台手机上,程序死活安装不上去,抓到Log:

01-30 15:17:33.801 3109-3242/? D/PackageManager: return install result to caller: 1141892560

01-30 15:17:33.801 3109-3242/? D/PackageManager: returnCode: -18

找到PackageManager.java的源码:https://android.googlesource.com/platform/frameworks/base/+/534a67c/core/java/android/content/pm/PackageManager.java

    // ------ Errors related to sdcard
    /**
     * Installation return code: this is passed to the {@link IPackageInstallObserver} by
     * {@link #installPackage(android.net.Uri, IPackageInstallObserver, int)} if
     * a secure container mount point couldn't be accessed on external media.
     * @hide
     */
    public static final int INSTALL_FAILED_CONTAINER_ERROR = -18;

然后在google上搜索 INSTALL_FAILED_CONTAINER_ERROR
http://www.cnblogs.com/lovecode/articles/3658211.html

https://stackoverflow.com/questions/5744298/what-does-this-mean-failure-install-failed-container-error

获得解决方案:


Change the install location from

android:installLocation="preferExternal"

into

android:installLocation="auto"

in your project's AndroidManifest.xml.

修改后重新出包,安装,成功。

pkm图片转为png

android sdk下包含一个小工具:

etc1tool.exe 可以将pkm格式的文件转为png.

命令格式:

etc1tool.exe  1.pkm –decode -o 1.png

rust 使用 protobuf

没有技术含量的东西,就是记录一下流水帐。

crate.io 上搜索 protobuf:

https://github.com/stepancheg/rust-protobuf

1、下载 google 的 protoc 解压后添加至环境变量

https://github.com/google/protobuf/releases

2、安装 protoc-gen-rust,可以使用命令:

cargo install protobuf

会安装在 C:\Users\Administrator\.cargo\bin 目录下,同样添加至环境变量。

3、使用protoc生成对应的 rust 原码文件

protoc --rust_out . foo.proto

4、在rust工程中Cargo.toml中的添加protobuf

[dependencies]
protobuf = "1.3.1"

5、添加引用的crate:

extern crate protobuf;

6、测试代码:

extern crate protobuf;
use protobuf::Message;

let mut hello=common::Hello::new();
hello.set_id(4);

let data=protobuf::Message::write_to_bytes(&hello).expect("error");
let data2=hello.write_to_bytes().expect("error");
println!("{:?}",data);
println!("{:?}",data2);

let hello=protobuf::parse_from_bytes::<common::Hello>(&data).expect("error");
println!("{}",hello.get_id());

rust学习第一课

使用 tokio 根据文档写一个收发字符串内容的tcp server.

文档地址:https://tokio.rs/docs/getting-started/simple-server/

#[macro_use]
extern crate log;

extern crate bytes;
extern crate tokio_io;
extern crate tokio_core;
extern crate tokio_proto;
extern crate tokio_service;
extern crate  byteorder;
extern crate futures;


use std::io;
use std::str;
use std::io::Cursor;
use bytes::{BytesMut,BufMut};
use tokio_io::codec::*;
use byteorder::*;
use tokio_proto::TcpServer;
use tokio_proto::pipeline::ServerProto;
//use tokio_proto::multiplex::ServerProto;
//use tokio_proto::streaming::pipeline::ServerProto;
use tokio_io::*;
use tokio_io::codec::Framed;
use tokio_service::Service;
use futures::{future,Future,BoxFuture};

#[test]
fn hello(){
    info!("hello");

    pub struct MessageCodec;

    impl Decoder for MessageCodec{
        type Item=String;
        type Error=io::Error;
        fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error>{
            println!("decode:{}",buf.len());
            if buf.len()<=4 {
                return Ok(None);
            }


            let len=BigEndian::read_i32(buf.split_to(4).as_ref()) as usize;

            let b=buf.split_to(len);
            let content=str::from_utf8(b.as_ref());
            println!("decode 2:{}",buf.len());
            return Ok( Some( content.unwrap().to_owned()  ));
        }
    }

    impl Encoder for MessageCodec{
        type Item=String;
        type Error=io::Error;
        fn encode(&mut self, item: String, buf: &mut BytesMut)
                  -> Result<(), Self::Error>{
            println!("encode:{}",item);
            buf.put_i32::<BigEndian>(item.len() as i32 );
            buf.put_slice(item.into_bytes().as_ref());
            return Ok(());
        }
    }

    pub struct MessageProto;

    impl<T:AsyncRead+AsyncWrite+'static> ServerProto<T> for MessageProto{
        type Request=String;
        type Response=String;
        type Transport=Framed<T,MessageCodec>;

        type BindTransport = Result<Self::Transport,io::Error>;
        fn bind_transport(&self, io: T) -> Self::BindTransport{
            Ok(io.framed(MessageCodec))
        }
    }

    pub struct GameServer;

    impl Service for GameServer{
        /// Requests handled by the service.
        type Request=String;

        /// Responses given by the service.
        type Response=String;

        /// Errors produced by the service.
        type Error=io::Error;

        /// The future response value.
        type Future=BoxFuture<Self::Response,Self::Error  >;

        /// Process the request and return the response asynchronously.
        fn call(&self, req: Self::Request) -> Self::Future{
            println!("receive:{}",req);

            return future::ok(req.chars().rev().collect()).boxed();
        }
    }

    let addr="0.0.0.0:9999".parse().unwrap();
    let server=TcpServer::new(MessageProto,addr);

    println!("start running");
    server.serve(|| Ok(GameServer));

    println!("hello test");
}

第一次基础上算正规的边看文档,边写,边理解语法,边google。

过程中掌握以下知识点:

1、类型转换
i32 转为 u32
a as u32
2、从 [u8] 中读取一个基础类型
BigEndian::read_i32(buf)
3、byte[] 转为字符串
       str::from_utf8([u8])
4、字符串转为bytes
str.into_bytes()
5、slice目前ide无法做到智能语法提示。
       需要自己来处理。
6、bytes取一个slice再转为byte[]
       buf[a..b].as_ref()

下一步就是自己用tcpstream去连上。

AssetBundle解析音频文件

使用 Unity Studio 解析 ab 时,发现在 obb 中的 ab 未解析出来。

disunity 又N年未更新了。

用 ABE 发现可以提取出 ab data和dump信息,但是工具无自动导出音频功能。

只能自己手动处理下了。

1、obb本身是压缩的ab格式,需要先将其解压,用 Unity Studio 解压即可。

2、对于这部分文件 UnityStudio无法正常解析(有时间可以看下源码找下原因),使用 UAB 打开后,将AudioClip 提取出 raw data(文件名类似: Select-voice-02_AudioClip_117_1854432161) ,提取出 raw dump (文件名类似:Raw_117_1854432161.dat)

我们需要做的事件就是将 ab的文件头从 raw dump中去除,即可得到音频文件。

0 AudioClip Base
 1 string m_Name = "Select_voice_02"
 0 int m_Format = 2
 0 int m_Type = 13
 0 bool m_3D = false
 1 bool m_UseHardware = false
 0 int m_Stream = 2
 0 vector m_AudioData
  1 Array Array (95608 items)

raw data 中列明了文件头信息,对我们有用的是 Array Array 行,指明了 音频文件的大小长度。

之后只需要从 raw dump中取最后 arraylength+1 字节内容即为音频内容。

三个story工具

http://www.nwn2toolset.dayjo.org/ToolsetTuts/conversations/writingconvos.html

https://twinery.org/

 

http://www.inklestudios.com/inklewriter/

Json web token (JWT) 相关介绍

最近看了vuejs,看到了vue-auth中一个jwt-auth,发现几年不关注,前端完全是全新的领域了。

这里列下jwt相关的资源,供快速了解jwt.

https://jwt.io/introduction/

https://github.com/szerhusenBC/jwt-spring-security-demo

https://www.toptal.com/java/rest-security-with-jwt-spring-security-and-java

spring boot security如何集成JWT.

java相关的jwt库太多了。

apache有oltu

spring有spring-security-jwt

还有其它一些第三方开源库

1、服务器端添加 jwt 支持:

配置spring security

package com.elex.gm.spring;

import java.security.SecureRandom;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.builders.WebSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
import org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter;

import com.elex.gm.service.UserManager;
import com.elex.gm.spring.jwt.JWTAuthenticationFilter;
import com.elex.gm.spring.jwt.JWTLoginFilter;

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {

    @Bean
    public BCryptPasswordEncoder passwordEncoder() throws Exception {
        return passwordEncoder;
    }

    public void configure(WebSecurity web) throws Exception {
        web.ignoring().antMatchers("/static/**").antMatchers("/hello/**").antMatchers("/api/open/**");
    };

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http

                .authorizeRequests()

                .antMatchers("/api/gm/**").hasAnyRole("ADMIN", "USER")

                .antMatchers("/api/user/**").hasAnyRole("USER")

                .and()

                .formLogin()

                .loginProcessingUrl("/api/user/login")

                .permitAll()

                .and()

                .logout()

                // .logoutUrl("/api/open/logout")

                .and()

                .httpBasic()

                .and()

                .addFilterBefore(new JWTLoginFilter("/api/user/login", authenticationManager()),
                        UsernamePasswordAuthenticationFilter.class)

                .addFilterBefore(new JWTAuthenticationFilter(), UsernamePasswordAuthenticationFilter.class);

        http.csrf().disable();
        http.headers().frameOptions().disable();
    }

    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.userDetailsService(userManager).passwordEncoder(passwordEncoder);
    }

    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception {
    }

    @Autowired
    private UserManager userManager;

    private BCryptPasswordEncoder passwordEncoder = new BCryptPasswordEncoder(16,
            new SecureRandom("j^E1)y*)HL2gT,es+ODa!0+j^L1I3+6I".getBytes()));
}

 

JWTLoginFilter:

package com.elex.gm.spring.jwt;

import java.io.IOException;
import java.util.Map;

import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.springframework.http.HttpStatus;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.AuthenticationException;
import org.springframework.security.jwt.Jwt;
import org.springframework.security.jwt.JwtHelper;
import org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter;

import com.alibaba.fastjson.JSON;
import com.elex.gm.model.User;
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;

public class JWTLoginFilter extends AbstractAuthenticationProcessingFilter {

    public JWTLoginFilter(String defaultFilterProcessesUrl, AuthenticationManager authenticationManager) {
        super(defaultFilterProcessesUrl);
        setAuthenticationManager(authenticationManager);
    }

    @Override
    public Authentication attemptAuthentication(HttpServletRequest request, HttpServletResponse response)
            throws AuthenticationException, IOException, ServletException {

        String username = request.getParameter("username");
        String password = request.getParameter("password");

        if (username == null) {
            username = "";
        } else {
            username = username.trim();
        }
        if (password == null) {
            password = "";
        } else {
            password = password.trim();
        }
        return getAuthenticationManager()
                .authenticate(new UsernamePasswordAuthenticationToken(username, password, Lists.newArrayList()));
    }

    @Override
    protected void successfulAuthentication(HttpServletRequest request, HttpServletResponse response, FilterChain chain,
            Authentication authResult) throws IOException, ServletException {
        User user = (User) authResult.getPrincipal();

        Map<String, Object> tokenMap = Maps.newHashMap();
        tokenMap.put("id", user.getId());
        tokenMap.put("name", user.getUsername());
        tokenMap.put("roles", user.getRoles());
        Jwt jwt = JwtHelper.encode(JSON.toJSONString(tokenMap), JWTKey.hmac);

        String token = jwt.getEncoded();
        response.setHeader("Authorization", token);
    }

    @Override
    protected void unsuccessfulAuthentication(HttpServletRequest request, HttpServletResponse response,
            AuthenticationException failed) throws IOException, ServletException {
        response.setStatus(HttpStatus.UNAUTHORIZED.value());
    }

}

JWTAuthentication:

package com.elex.gm.spring.jwt;

import java.io.IOException;
import java.util.List;

import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;

import org.apache.commons.lang3.StringUtils;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.jwt.Jwt;
import org.springframework.security.jwt.JwtHelper;
import org.springframework.web.filter.GenericFilterBean;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import com.google.common.collect.Lists;

public class JWTAuthenticationFilter extends GenericFilterBean {
    private static final String JWT_HEADER = "Authorization";

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {
        try {
            String token = ((HttpServletRequest) request).getHeader(JWT_HEADER);
            Jwt jwt = JwtHelper.decodeAndVerify(token, JWTKey.hmac);
            JSONObject json = JSON.parseObject(jwt.getClaims());

            String username = json.getString("name");
            String roles = json.getString("roles");
            List<SimpleGrantedAuthority> list = Lists.newArrayList();
            if (!StringUtils.isEmpty(roles)) {
                for (String role : roles.split(",")) {
                    if (!StringUtils.isEmpty(role)) {
                        list.add(new SimpleGrantedAuthority("ROLE_" + role));
                    }
                }
            }
            Authentication authentication = new UsernamePasswordAuthenticationToken(username, null, list);
            SecurityContextHolder.getContext().setAuthentication(authentication);
        } catch (Exception e) {
            // e.printStackTrace();
        }
        chain.doFilter(request, response);
    }
}

如何判断是否为中国IP

收到一个需求,需要判断IP是否来自中国。

判断IP来源,很早之前自己也感兴趣,一直没有去了解如何实现。

今天正好搜索了一下相关的文章。

亚洲所有相关分配的IP段可以从 http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest 查询。

从里面解释后即可实现判断IP是否为中国。

没有找到java的代码,就自己写了一份,供其它朋友使用。

将下载下来的 delegated-apnic-latest 放在 classpath下即可。

import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Collections;
import java.util.List;
import java.util.Map;

import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import org.assertj.core.util.Lists;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.google.common.collect.Maps;

public class IpUtil {
    private static final String FILE_NAME = "delegated-apnic-latest";

    // 只存放属于中国的ip段
    private static Map<Integer, List<int[]>> chinaIps = Maps.newHashMap();
    static {
        init();
    }

    public static void init() {
        try {
            // ip格式: add1.add2.add3.add4
            // key为 : add1*256+add2
            // value为int[2]: int[0]存的add3*256+add4的开始ip int[4]存的结束ip
            Map<Integer, List<int[]>> map = Maps.newHashMap();

            InputStream input = Thread.currentThread().getContextClassLoader().getResourceAsStream(FILE_NAME);
            List<String> lines = IOUtils.readLines(input, StandardCharsets.UTF_8);
            for (String line : lines) {
                if (line.startsWith("apnic|CN|ipv4|")) {
                    // 只处理属于中国的ipv4地址
                    String[] strs = line.split("\\|");
                    String ip = strs[3];
                    String[] add = ip.split("\\.");
                    int count = Integer.valueOf(strs[4]);

                    int startIp = Integer.parseInt(add[0]) * 256 + Integer.parseInt(add[1]);
                    while (count > 0) {
                        if (count >= 65536) {
                            // add1,add2 整段都是中国ip
                            chinaIps.put(startIp, Collections.EMPTY_LIST);
                            count -= 65536;
                            startIp += 1;
                        } else {

                            int[] ipRange = new int[2];
                            ipRange[0] = Integer.parseInt(add[2]) * 256 + Integer.parseInt(add[3]);
                            ipRange[1] = ipRange[0] + count;
                            count -= count;

                            List<int[]> list = map.get(startIp);
                            if (list == null) {
                                list = Lists.newArrayList();
                                map.put(startIp, list);
                            }

                            list.add(ipRange);
                        }
                    }
                }
            }
            chinaIps = map;
        } catch (Exception e) {
            logger.error("ERROR", e);
        }
    }

    public static boolean isChinaIp(String ip) {
        if (StringUtils.isEmpty(ip)) {
            return false;
        }
        String[] strs = ip.split("\\.");
        if (strs.length != 4) {
            return false;
        }
        int key = Integer.valueOf(strs[0]) * 256 + Integer.valueOf(strs[1]);
        List<int[]> list = chinaIps.get(key);
        if (list == null) {
            return false;
        }
        if (list.size() == 0) {
            // 整段都是中国ip
            return true;
        }
        int ipValue = Integer.valueOf(strs[2]) * 256 + Integer.valueOf(strs[3]);
        for (int[] ipRange : list) {
            if (ipValue >= ipRange[0] && ipValue <= ipRange[1]) {
                return true;
            }
        }

        return false;
    }

    private static final Logger logger = LoggerFactory.getLogger(IpUtil.class);
}

 

如果扩展这个代码后,可以增加判断IP属于哪个国家。

如果在判断IP属于哪个城市,可能得需要 http://dev.maxmind.com/zh-hans/geoip/geoip2/

 

参考:

https://www.zhihu.com/question/19794443

http://xixitalk.github.io/blog/2013/03/11/func-is-china-ip/