#AF

KAFKA:如何做到1秒发布百万级条消息

http://rdcqii.hundsun.com/portal/article/709.htmlKAFKA是分布式发布-订阅消息系统,是一个分布式的,可划分的,冗余备份的持久性的日志服务。它主要用于处理活跃的流式数据。现在被广泛地应用于构建实时数据管道和流应用的场景中,具有横向扩展,容错,快等优点,并已经运行在众多大...

Kafka生产者性能优化之吞吐量VS延迟

WhenwearetalkingaboutperformanceofKafkaProducer,wearereallytalkingabouttwodifferentthings:latency:howmuchtimepassesfromthetimeKafkaProducer.send()wascalleduntil...

Debugging Kafka connect

1.setupdebugconfigurationmainClass:org.apache.kafka.connect.cli.ConnectDistributedVMOption:-Dkafka.logs.dir=/home/lenmom/workspace/software/kafka_2.11-2.1.0/bin...
代码星球 ·2020-04-05

kafka shell file

1.startkafkaandschema_registry #!/bin/shexportKAFKA_HOME=/home/lenmom/workspace/software/confluent-community-5.1.0-2.11exportKAFKA_PROCESS_ID=$(jps|grepSup...
代码星球 ·2020-04-05

confluent kafka connect remote debugging

Tobeginwith,let'stakealookathowkafkaconnectstart.1.1 startcommand#backgroundrunningmodecd/home/lenmom/workspace/software/confluent-community-5.1.0-2.11/&am...

Kafka Connect REST Interface

SinceKafkaConnectisintendedtoberunasaservice,italsosupportsaRESTAPIformanagingconnectors.Bydefaultthisservicerunsonport 8083.Whenexecutedindistributedmode,...
代码星球 ·2020-04-05

building confluentinc kafka-connect-hdfs

WhenItrytocompileIgetanerroraboutamissingSNAPSHOTdependency.Theerrorlookssomethinglikethis:Failedtoexecutegoalonprojectkafka-connect-hdfs:Couldnotresolvedepende...

using kafkacat reset kafka offset

1.installkafkacatUbuntuapt-getinstallkafkacatCentOSinstalldeepenencyyuminstalllibrdkafka-develdownloadsourcefromgithubbuildsourceoncentos./configure<usual-co...

Kafka Connect Architecture

KafkaConnect'sgoalofcopyingdatabetweensystemshasbeentackledbyavarietyofframeworks,manyofthemstillactivelydevelopedandmaintained.Thissectionexplainsthemotivation...
代码星球 ·2020-04-05

The usage of docker image wurstmeister/kafka

 Thedockerimage wurstmeister/kafkaisthemoststaredimageforkafkainhub.docker.com,buttheuseageisrare,so inthispost,Iwouldtakesometimetotalkabouttheu...
代码星球 ·2020-04-05

kafka connect rest api

1.获取ConnectWorker信息curl-shttp://127.0.0.1:8083/|jqlenmom@M1701:~/workspace/software/kafka_2.11-2.1.0/logs$curl-shttp://127.0.0.1:8083/|jq{"version":"2.1.0","com...
代码星球 ·2020-04-05

Kafka connect in practice(3): distributed mode mysql binlog ->kafka->hive

InthepreviouspostKafkaconnectinpractice(1):standalone,Ihaveintroducedaboutthebasicsofkafkaconnect  configurationanddemonstratealocalstandalonedemo.Int...

Linux Kafka源码环境搭建

本文主要讲述的是如何搭建Kafka的源码环境,主要针对的Linux操作系统下IntelliJIDEA编译器,其余操作系统或者IDE可以类推。1.安装和配置JDK确认JDK版本至少为1.7,最好是1.8及以上。使用java-version命令来查看当前JDK的版本,示例如下:lenmom@M1701:~/workspac...

kafka producer实例

1.定义要发送的消息UserPOJOpackagelenmom.kafkaproducer;publicclassUser{publicStringname;publicintage;publicStringaddress;publicUser(Stringname,intage,Stringaddress){this...
代码星球 ·2020-04-05

CentOS7 安装kafka集群

1.环境准备JDK1.8ZooKeeper集群(参见本人博文)Scala2.12(如果需要做scala开发的话,安装方法参见本人博文)本次安装的kafka和zookeeper集群在同一套物理机器上,192.168.1.101192.168.1.102192.168.1.103 2.下载kafka到kafka官...
代码星球 ·2020-04-05
首页上一页...4849505152...下一页尾页