技术探索WebRTC实现本地视频通话 | 全流程详解
BreezliWebRTC 实现本地视频通话 | 全流程详解
源码见仓库 Breezli/WebRTC_Demo: 基于 WebRTC 技术实现的本地网页端视频通话
分为前后端两个文件夹
webrtc-client (客户端)
webrtc-server (服务端)
预先准备
webrtc-client
前端页面绘制
依赖下载
1 2 3
| pnpm create vite@latest webrtc-client -- --template vue-ts pnpm install -D tailwindcss postcss autoprefixer npx tailwindcss init -p
|
package.json
直通
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
| { "name": "webrtc-client", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "vite", "build": "vue-tsc -b && vite build", "preview": "vite preview" }, "dependencies": { "socket.io-client": "^4.8.1", "vue": "^3.5.13" }, "devDependencies": { "@vitejs/plugin-vue": "^5.2.1", "@vue/tsconfig": "^0.7.0", "autoprefixer": "^10.4.20", "postcss": "^8.4.49", "tailwindcss": "^3.4.17", "typescript": "~5.6.2", "vite": "^6.0.5", "vue-tsc": "^2.2.0" } }
|
App.vue
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
| <template> <div class="flex items-center flex-col text-center p-12 h-screen"> <div class="relative h-full mb-4"> <video ref="localVideo" class="w-96 h-full bg-gray-200 mb-4 object-cover"></video> <video ref="remoteVideo" class="w-32 h-48 absolute bottom-0 right-0 object-cover"></video> <div v-if="caller && calling" class="absolute top-2/3 left-36 flex flex-col items-center"> <p class="mb-4 text-white">等待对方接听...</p> <img @click="hangUp" src="/refuse.svg" class="w-16 cursor-pointer" alt="" /> </div> <div v-if="called && calling" class="absolute top-2/3 left-32 flex flex-col items-center"> <p class="mb-4 text-white">收到视频邀请...</p> <div class="flex"> <img @click="hangUp" src="/refuse.svg" class="w-16 cursor-pointer mr-4" alt="" /> <img @click="acceptCall" src="/accept.svg" class="w-16 cursor-pointer" alt="" /> </div> </div> </div> <div class="flex gap-2 mb-4"> <button class="rounded-md bg-indigo-600 px-4 py-2 text-sm font-semibold text-white" @click="callRemote"> 发起视频 </button> <button class="rounded-md bg-red-600 px-4 py-2 text-sm font-semibold text-white" @click="hangUp"> 挂断视频 </button> </div> </div> </template>
|
main.ts
1 2 3 4 5
| import { createApp } from 'vue' import './style.css' import App from './App.vue'
createApp(App).mount('#app')
|
style.css
这里使用了 tailwind 的样式库
1 2 3
| @tailwind base; @tailwind components; @tailwind utilities;
|
tailwind.config.js
此配置文件中添加所有模板文件的路径
1 2 3 4 5 6 7 8
| export default { content: ['./index.html', './src/**/*.{vue,js,ts,jsx,tsx}'], theme: { extend: {}, }, plugins: [], }
|
到此前端页面绘制完成,可在 3000 端口可启动 node 服务
webrtc-server
初始化服务端项目
socket.io
封装了 websocket 应用创建服务
nodemon
代码改变会自动重启(不用重复地开关)
1
| pnpm install socket.io nodemon
|
package.json
添加start
命令,使用nodemon
启动项目
1 2 3 4
| "scripts": { "test": "echo \"Error: no test specified\" && exit 1", "start": "nodemon index.js" },
|
创建index.js
(最初版,之后后端的所有操作都在这个文件上进行)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
| const socket = require('socket.io') const http = require('http')
const server = http.createServer()
const io = socket(server, { cors: { origin: '*', }, })
io.on('connection', (sock) => { console.log('连接成功...') sock.emit('connectionSuccess') })
io.on('disconnect', () => { console.log('连接断开') sock.emit('connectionFalse') })
server.listen(3000, () => { console.log('服务器启动成功') })
|
!!!插播 socket 重要概念!!!
常见内置事件说明
事件名称 |
触发场景 |
使用位置 |
connect |
客户端成功连接到服务端时 |
客户端 |
connection |
服务端接收到新客户端连接时 |
服务端 |
disconnect |
连接断开时 |
客户端/服务端 |
error |
连接发生错误时 |
客户端/服务端 |
后端至此搭建完成,可在 3000 端口可启动 node 服务
前端连接信令服务器
依靠的就是socket.io-client
接下来我们将在App.vue
中开始编写脚本
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
| <script setup lang="ts"> import { ref, onMounted, onUnmounted } from 'vue' import { io, Socket } from 'socket.io-client'
const socket = ref<Socket>() // 前端Socket实例
onMounted(() => { const sock = io('localhost:3000') // 通过 io('localhost:3000') 创建的 Socket.IO 客户端实例
// 连接成功 sock.on('connectionSuccess', () => { console.log('连接成功') })
socket.value = sock // 存储Socket实例 }) </script>
|
还记得吗,就在前两步,后端在连接成功后向前端.emit(发送)了 connectionSuccess 事件
1 2 3 4
| io.on('connection', (sock) => { console.log('连接成功...') sock.emit('connectionSuccess') })
|
而在此时的前端加入的
1 2 3 4
| sock.on('connectionSuccess', () => { console.log('连接成功') })
|
前后端交互正式开始
发起视频请求
获取本地音视频流
前端
1 2 3 4 5 6 7 8 9 10 11 12
| const getLocalStream = async () => { const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true, }) localVideo.value!.srcObject = stream localVideo.value!.play() localStream.value = stream
return stream }
|
“房间”的概念
不妨先停下来思考一下,为什么在同一个网页,不同的客户端可以看见不同的页面,和实时共享的内容
我们先来看一看之前给前端写的组件(layout 省略版)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
| <video ref="localVideo"></video> //本地视频 <video ref="remoteVideo"></video> //接收方视频(位于右下角)
<div v-if="caller && calling"> //状态参数---caller:发送方---calling:呼叫中 <p>等待对方接听...</p> <img @click="hangUp" src="/refuse.svg"/> //挂断 </div>
<div v-if="called && calling"> //状态参数---called:接收方---calling:呼叫中 <p>收到视频邀请...</p>
<img @click="hangUp" src="/refuse.svg"/> //挂断 <img @click="acceptCall" src="/accept.svg"/>//接听
<button @click="callRemote">发起视频</button> <button @click="hangUp">挂断视频</button> </div>
|
不难发现我们是依靠状态参数来控制页面显示的
本项目参数表如下
1 2 3 4 5 6 7 8 9
| const called = ref<boolean>(false) const caller = ref<boolean>(false) const calling = ref<boolean>(false) const communicating = ref<boolean>(false) const localVideo = ref<HTMLVideoElement>() const remoteVideo = ref<HTMLVideoElement>() const localStream = ref<MediaStream>()
const roomId = '001'
|
顺便带上实例创建
1 2
| const socket = ref<Socket>() const peer = ref<any>()
|
而对于我们这个项目,房间
的概念就很简单
A (左) 向 B (右) 发起接通请求,在 B 接收请求后,双方都将加入这个视频通话的”房间”
房间 可以通过自主分配 id 划分,每个房间也可以加入很多用户 (如直播系统)
服务端含有加入房间的操纵代码 sock.join(Id)
加入房间请求
前端
1 2 3 4
| sock.on('connectionSuccess', () => { console.log('连接服务器成功...') sock.emit('joinRoom', roomId) })
|
后端
1 2 3
| sock.on('joinRoom', (roomId) => { sock.join(roomId) })
|
发起视频请求
A 方按钮触发函数
前端
1 2 3 4 5 6 7
| const callRemote = async () => { console.log('发起视频') caller.value = true calling.value = true await getLocalStream() socket.value?.emit('callRemote', roomId) }
|
后端
1 2 3 4
| sock.on('callRemote', (roomId) => { io.to(roomId).emit('receiveCall') })
|
前端
1 2 3 4 5 6 7 8
| sock.on('receiveCall', () => { if (!caller.value) { calling.value = true called.value = true } })
|
此时 A(发起)方启动摄像头和麦克风,并等待对方接听
B(接收)方出现 接听/挂断 按钮
1 2 3 4 5 6 7 8 9 10 11 12 13 14
| <div v-if="caller && calling"> //状态参数---caller:发送方---calling:呼叫中 <p>等待对方接听...</p> <img @click="hangUp" src="/refuse.svg"/> //挂断 </div>
<div v-if="called && calling"> //状态参数---called:接收方---calling:呼叫中 <p>收到视频邀请...</p>
<img @click="hangUp" src="/refuse.svg"/> //挂断 <img @click="acceptCall" src="/accept.svg"/>//接听
<button @click="callRemote">发起视频</button>//发起 <button @click="hangUp">挂断视频</button> //挂断 </div>
|
接收 / 挂断 视频
趁着现在,我们添上其他按钮绑定的函数
1 2 3 4
| const hangUp = () => { socket.value.emit('hangUp', roomId) }
|
1 2 3 4
| const acceptCall = () => { socket.value.emit('acceptCall', roomId) }
|
接着来看后端怎么处理
1 2 3 4
| sock.on('hangUp', (roomId) => { io.to(roomId).emit('hangUp') })
|
1 2 3 4
| sock.on('acceptCall', (roomId) => { io.to(roomId).emit('acceptCall') })
|
返回前端,处理我们接收到的 hangUp || acceptCall
挂断视频
1 2 3
| sock.on('hangUp', () => { reset() })
|
重置状态函数
1 2 3 4 5 6 7 8 9 10 11
| const reset = () => { called.value = false caller.value = false calling.value = false communicating.value = false peer.value = null localVideo.value!.srcObject = null remoteVideo.value!.srcObject = null localStream.value?.getTracks()[0].stop() }
|
接受视频请求
而这个就变复杂了
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
| sock.on('acceptCall', async () => { if (caller.value) { peer.value = new RTCPeerConnection() peer.value.addStream(localStream.value)
peer.value.onicecandidate = (event: any) => { if (event.candidate) { sock.emit('sendCandidate', { roomId, candidate: event.candidate }) } } peer.value.onaddstream = (event: any) => { calling.value = false communicating.value = true remoteVideo.value!.srcObject = event.stream remoteVideo.value!.play() } const offer = await peer.value.createOffer({ offerToReceiveAudio: true, offerToReceiveVideo: true, }) await peer.value.setLocalDescription(offer) sock.emit('sendOffer', { roomId, offer }) } })
|
交换 SDP 信息和 candidate 信息
概念
SDP :WebRTC 的一种协议,用于描述设备支持的媒体格式
candidate :网络信息
媒体协商 :交换 SDP
ICE :WebRTC 的通信机制
工作原理
1.双方收集本地网络地址(分 共有 | 私有)+ STUN 和 TURN 服务器获取候选地址
2.双方通过信令服务器交换这些候选地址
3.双方使用候选地址进行连接测试,确定最佳的可用地址,开始实时音视频通话
媒体协商
接下来是最重要的信令交换步骤(WebRTC 核心)
主要用到以下方法
媒体协商
createOffer
createAnswer
setLocalDesccription
setRemoteDesccription
重要事件
onicecandidate
onaddstream
步骤流程重绘(可以对标号有个印象)
预告:所有函数均在前端实现,后端起到向前端广播作用
紧接着接收方接收到对方同意的消息
信息交换全程
前端(发送方)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
| sock.on('acceptCall', async () => { if (caller.value) { peer.value = new RTCPeerConnection() peer.value.addStream(localStream.value)
peer.value.onicecandidate = (event: any) => { if (event.candidate) { sock.emit('sendCandidate', { roomId, candidate: event.candidate }) } }
peer.value.onaddstream = (event: any) => { calling.value = false communicating.value = true remoteVideo.value!.srcObject = event.stream remoteVideo.value!.play() }
---①--- const offer = await peer.value.createOffer({ offerToReceiveAudio: true, offerToReceiveVideo: true, }) ---②--- await peer.value.setLocalDescription(offer) ---③--- sock.emit('sendOffer', { roomId, offer }) } })
|
后端
1 2 3
| sock.on('sendCandidate', ({ roomId, candidate }) => { io.to(roomId).emit('receiveCandidate', candidate) })
|
1 2 3 4
| sock.on('sendOffer', ({ roomId, offer }) => { io.to(roomId).emit('sendOffer', offer) })
|
前端(接收方)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
| sock.on('sendOffer', async (offer: any) => { if (called.value) { const stream = await getLocalStream() peer.value = new RTCPeerConnection() peer.value.addStream(stream)
peer.value.onicecandidate = (event: any) => { if (event.candidate) { sock.emit('sendCandidate', { roomId, candidate: event.candidate }) } }
peer.value.onaddstream = (event: any) => { calling.value = false communicating.value = true remoteVideo.value!.srcObject = event.stream remoteVideo.value!.play() }
---④--- await peer.value.setRemoteDescription(offer)
---⑤--- const answer = await peer.value.createAnswer() ---⑥--- await peer.value.setLocalDescription(answer) ---⑦--- sock.emit('sendAnswer', { roomId, answer }) } })
|
后端
1 2 3 4
| sock.on('sendCandidate', ({ roomId, candidate }) => { io.to(roomId).emit('receiveCandidate', candidate) })
|
1 2 3
| sock.on('sendAnswer', ({ roomId, answer }) => { io.to(roomId).emit('receiveAnswer', answer) })
|
前端
1 2 3 4 5 6
| sock.on('receiveAnswer', (answer: any) => { if (caller.value) { ---⑧--- peer.value.setRemoteDescription(answer) } })
|
为了让文章不那么乱,最后再补一下candidate
1 2 3
| sock.on('receiveCandidate', async (candidate: any) => { await peer.value.addIceCandidate(candidate) })
|
一路下来是不是逻辑非常清晰明了 😁 (bushi)
完整代码
前端 (App.vue)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185
| import { ref, onMounted } from 'vue' import { io, Socket } from 'socket.io-client'
const roomId = '001'
const called = ref<boolean>(false) const caller = ref<boolean>(false) const calling = ref<boolean>(false) const communicating = ref<boolean>(false) const localVideo = ref<HTMLVideoElement>() const remoteVideo = ref<HTMLVideoElement>() const localStream = ref<MediaStream>() const socket = ref<Socket>() const peer = ref<any>()
onMounted(() => { const sock = io('localhost:3001')
sock.on('connectionSuccess', () => { sock.emit('joinRoom', roomId) })
sock.on('receiveCall', () => { if (!caller.value) { calling.value = true called.value = true } })
sock.on('acceptCall', async () => { if (caller.value) { peer.value = new RTCPeerConnection() peer.value.addStream(localStream.value)
peer.value.onicecandidate = (event: any) => { if (event.candidate) { sock.emit('sendCandidate', { roomId, candidate: event.candidate }) } }
peer.value.onaddstream = (event: any) => { calling.value = false communicating.value = true remoteVideo.value!.srcObject = event.stream remoteVideo.value!.play() }
const offer = await peer.value.createOffer({ offerToReceiveAudio: true, offerToReceiveVideo: true, }) await peer.value.setLocalDescription(offer) sock.emit('sendOffer', { roomId, offer }) } })
sock.on('sendOffer', async (offer: any) => { if (called.value) { const stream = await getLocalStream()
peer.value = new RTCPeerConnection()
peer.value.addStream(stream)
peer.value.onicecandidate = (event: any) => { if (event.candidate) { sock.emit('sendCandidate', { roomId, candidate: event.candidate }) } }
peer.value.onaddstream = (event: any) => { calling.value = false communicating.value = true remoteVideo.value!.srcObject = event.stream remoteVideo.value!.play() }
await peer.value.setRemoteDescription(offer)
const answer = await peer.value.createAnswer() await peer.value.setLocalDescription(answer) sock.emit('sendAnswer', { roomId, answer }) } })
sock.on('receiveAnswer', (answer: any) => { if (caller.value) { peer.value.setRemoteDescription(answer) } })
sock.on('receiveCandidate', async (candidate: any) => { await peer.value.addIceCandidate(candidate) })
sock.on('hangUp', () => { reset() })
socket.value = sock })
const getLocalStream = async () => { const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true, }) localVideo.value!.srcObject = stream localVideo.value!.play() localStream.value = stream
return stream }
const callRemote = async () => { if (calling.value || communicating.value) { return } calling.value = true await getLocalStream()
caller.value = true socket.value.emit('callRemote', roomId) }
const acceptCall = () => { socket.value.emit('acceptCall', roomId) }
const hangUp = () => { socket.value.emit('hangUp', roomId) }
const reset = () => { called.value = false caller.value = false calling.value = false communicating.value = false peer.value = null localVideo.value!.srcObject = null remoteVideo.value!.srcObject = null localStream.value?.getTracks()[0].stop() }
|
后端 (index.js)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
| const socket = require('socket.io') const http = require('http')
const server = http.createServer()
const io = socket(server, { cors: { origin: '*', }, })
io.on('connection', (sock) => { console.log('连接成功...')
sock.emit('connectionSuccess')
sock.on('joinRoom', (roomId) => { sock.join(roomId) })
sock.on('callRemote', (roomId) => { io.to(roomId).emit('receiveCall') })
sock.on('acceptCall', (roomId) => { io.to(roomId).emit('acceptCall') })
sock.on('sendOffer', ({ roomId, offer }) => { io.to(roomId).emit('sendOffer', offer) })
sock.on('sendAnswer', ({ roomId, answer }) => { io.to(roomId).emit('receiveAnswer', answer) })
sock.on('sendCandidate', ({ roomId, candidate }) => { io.to(roomId).emit('receiveCandidate', candidate) })
sock.on('hangUp', (roomId) => { io.to(roomId).emit('hangUp') }) })
server.listen(3001, () => { console.log('服务器启动成功') })
|
项目启动方式
后端启动服务
前端
1 2
| pnpm install pnpm run dev
|
WebSocket+NestJS 升级项目 | 支持不同网络下视频通话
项目架构重构
1 2 3 4 5 6 7 8
| ├── webrtc-client/ # Vue 前端 │ ├── src/ │ │ ├── App.vue # 主界面 │ │ └── webrtc.ts # WebRTC 核心逻辑(从App.vue分离出来) ├── webrtc-server/ # NestJS 后端 │ ├── src/ │ │ ├── signaling/ # 信令服务器模块 │ │ └── main.ts # 入口文件
|