未分類
26 2 月 2021

express 压缩接口响应的数据量

express 压缩接口响应的数据量

資深大佬 : 6EyesFlyFish 2

express 接口假如响应的数据量达到 7Kb 的时候，在高并发压测的情况下，异常率很高，有没有什么办法压缩数据量？
响应数据如下：

{
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 22.95,
“content”: “ok”,
“item”: [{
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 107.22,
“nick”: “糖衣炮弹 54720”,
“rank”: “1”,
“score”: “180”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 37.99,
“nick”: “糖衣炮弹 54717”,
“rank”: “2”,
“score”: “100”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://thirdwx.qlogo.cn/mmopen/vi_32/qDNuYJXiaUZJSlZq2XnmaBDNIIUJBlmPGNZpvbbv6NHibLzIGwzKDmzvibZTd3B0rm7I5BQ0Tcc0aLicU027L7IDibg/132”,
“all_time”: 60.41,
“nick”: “一船星辉”,
“rank”: “3”,
“score”: “120”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 36.1,
“nick”: “54723”,
“rank”: “4”,
“score”: “90”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 43.72,
“nick”: “54725”,
“rank”: “5”,
“score”: “90”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 25.32,
“nick”: “54719”,
“rank”: “6”,
“score”: “70”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 28.51,
“nick”: “54713”,
“rank”: “7”,
“score”: “70”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 38.81,
“nick”: “54714”,
“rank”: “8”,
“score”: “80”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 21.8,
“nick”: “54718”,
“rank”: “9”,
“score”: “60”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 32.19,
“nick”: “54726”,
“rank”: “10”,
“score”: “70”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 18.34,
“nick”: “54716”,
“rank”: “11”,
“score”: “50”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 28.7,
“nick”: “54724”,
“rank”: “12”,
“score”: “60”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 19.37,
“nick”: “54727”,
“rank”: “13”,
“score”: “50”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 22.95,
“nick”: “54715”,
“rank”: “14”,
“score”: “50”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 34.49,
“nick”: “糖衣炮弹 oaAq05gIwIhVcT4ydGkqfqaodfdo11”,
“rank”: “15”,
“score”: “60”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 18.08,
“nick”: “54711”,
“rank”: “16”,
“score”: “40”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 19.53,
“nick”: “糖衣炮弹 54721”,
“rank”: “17”,
“score”: “40”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 9.78,
“nick”: “糖衣炮弹 54722”,
“rank”: “18”,
“score”: “20”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 4.53,
“nick”: “54710”,
“rank”: “19”,
“score”: “10”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 14.79,
“nick”: “糖衣炮弹”,
“rank”: “20”,
“score”: “20”,
“wrongcount”: “0”
}, {
“HeadImgUrl”: “https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fn.sinaimg.cn%2Fsinacn%2Fw800h452%2F20180216%2F1c14-fyrpeie9854000.jpg&refer=http%3A%2F%2Fn.sinaimg.cn&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1616832170&t=5ca0bf41e39ebd9656bfb20da45775fd”,
“all_time”: 14.74,
“nick”: “54712”,
“rank”: “21”,
“score”: “10”,
“wrongcount”: “0”
}],
“all_score”: “50”,
“nick”: “54715”,
“pNumber”: “166556”,
“rank”: “14”,
“status”: 2000,
“wrongcount”: “0”
}

大佬有話說 (11)

  • 資深大佬 : libook

    异常的具体信息发一下，可能跟数据量大小没关系。

  • 主 資深大佬 : 6EyesFlyFish

    异常信息是连接重置
    Response code:Non HTTP response code: java.net.SocketException
    Response message:Non HTTP response message: Connection reset

    我把数据量减少，再压测就没有异常了

  • 主 資深大佬 : 6EyesFlyFish

    @libook

  • 資深大佬 : libook

    Node 上、服务器系统级别、反向代理服务器等等，是否异常日志呢？
    以及服务器资源情况，负载、网络连接数、磁盘 IO 、内存使用量等等。

  • 資深大佬 : killergun

    我感觉这个是不是设置下服务 response 可响应大小就行了

  • 資深大佬 : killergun

    @killergun 还有就是设置响应超时时间

  • 主 資深大佬 : 6EyesFlyFish

    @libook 腾讯云服务器 8 核 16G，0 负载，8%内存使用，ulimit 设置了 100W，没有反向代理，异常日志没有，部分连接直接被重置了

  • 主 資深大佬 : 6EyesFlyFish

    @killergun express 可响应大小？在哪设置？我看了官网，似乎没有这方面的介绍

  • 資深大佬 : wakzz

    建议主抓个包，把异常 TCP 抓到后贴出来，单单通过主目前提供的数据推测原因有点困难

  • 主 資深大佬 : 6EyesFlyFish

    @wakzz 搞定了，网络带宽问题，数据量太大了

  • 資深大佬 : wakzz

    @6EyesFlyFish 这让我想起一次跟第三方联调时压测的网络错误问题，抓包后发现压测到某个阈值就有大量的网络超时，最后查明是对方公司的宽带不够，吐血