Skip to content

Commit 6e5ea7f

Browse files
pkamp3pkamp3
committed
Initial add license, code of conduct, package.json, server code
1 parent 21265fd commit 6e5ea7f

File tree

4 files changed

+353
-0
lines changed

4 files changed

+353
-0
lines changed

‎CODE_OF_CONDUCT.md

+134
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
2+
# Contributor Covenant Code of Conduct
3+
4+
## Our Pledge
5+
6+
We as members, contributors, and leaders pledge to make participation in our
7+
community a harassment-free experience for everyone, regardless of age, body
8+
size, visible or invisible disability, ethnicity, sex characteristics, gender
9+
identity and expression, level of experience, education, socio-economic status,
10+
nationality, personal appearance, race, caste, color, religion, or sexual
11+
identity and orientation.
12+
13+
We pledge to act and interact in ways that contribute to an open, welcoming,
14+
diverse, inclusive, and healthy community.
15+
16+
## Our Standards
17+
18+
Examples of behavior that contributes to a positive environment for our
19+
community include:
20+
21+
* Demonstrating empathy and kindness toward other people
22+
* Being respectful of differing opinions, viewpoints, and experiences
23+
* Giving and gracefully accepting constructive feedback
24+
* Accepting responsibility and apologizing to those affected by our mistakes,
25+
and learning from the experience
26+
* Focusing on what is best not just for us as individuals, but for the overall
27+
community
28+
29+
Examples of unacceptable behavior include:
30+
31+
* The use of sexualized language or imagery, and sexual attention or advances of
32+
any kind
33+
* Trolling, insulting or derogatory comments, and personal or political attacks
34+
* Public or private harassment
35+
* Publishing others' private information, such as a physical or email address,
36+
without their explicit permission
37+
* Other conduct which could reasonably be considered inappropriate in a
38+
professional setting
39+
40+
## Enforcement Responsibilities
41+
42+
Community leaders are responsible for clarifying and enforcing our standards of
43+
acceptable behavior and will take appropriate and fair corrective action in
44+
response to any behavior that they deem inappropriate, threatening, offensive,
45+
or harmful.
46+
47+
Community leaders have the right and responsibility to remove, edit, or reject
48+
comments, commits, code, wiki edits, issues, and other contributions that are
49+
not aligned to this Code of Conduct, and will communicate reasons for moderation
50+
decisions when appropriate.
51+
52+
## Scope
53+
54+
This Code of Conduct applies within all community spaces, and also applies when
55+
an individual is officially representing the community in public spaces.
56+
Examples of representing our community include using an official email address,
57+
posting via an official social media account, or acting as an appointed
58+
representative at an online or offline event.
59+
60+
## Enforcement
61+
62+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
63+
reported to the community leaders responsible for enforcement at
64+
[email protected].
65+
All complaints will be reviewed and investigated promptly and fairly.
66+
67+
All community leaders are obligated to respect the privacy and security of the
68+
reporter of any incident.
69+
70+
## Enforcement Guidelines
71+
72+
Community leaders will follow these Community Impact Guidelines in determining
73+
the consequences for any action they deem in violation of this Code of Conduct:
74+
75+
### 1. Correction
76+
77+
**Community Impact**: Use of inappropriate language or other behavior deemed
78+
unprofessional or unwelcome in the community.
79+
80+
**Consequence**: A private, written warning from community leaders, providing
81+
clarity around the nature of the violation and an explanation of why the
82+
behavior was inappropriate. A public apology may be requested.
83+
84+
### 2. Warning
85+
86+
**Community Impact**: A violation through a single incident or series of
87+
actions.
88+
89+
**Consequence**: A warning with consequences for continued behavior. No
90+
interaction with the people involved, including unsolicited interaction with
91+
those enforcing the Code of Conduct, for a specified period of time. This
92+
includes avoiding interactions in community spaces as well as external channels
93+
like social media. Violating these terms may lead to a temporary or permanent
94+
ban.
95+
96+
### 3. Temporary Ban
97+
98+
**Community Impact**: A serious violation of community standards, including
99+
sustained inappropriate behavior.
100+
101+
**Consequence**: A temporary ban from any sort of interaction or public
102+
communication with the community for a specified period of time. No public or
103+
private interaction with the people involved, including unsolicited interaction
104+
with those enforcing the Code of Conduct, is allowed during this period.
105+
Violating these terms may lead to a permanent ban.
106+
107+
### 4. Permanent Ban
108+
109+
**Community Impact**: Demonstrating a pattern of violation of community
110+
standards, including sustained inappropriate behavior, harassment of an
111+
individual, or aggression toward or disparagement of classes of individuals.
112+
113+
**Consequence**: A permanent ban from any sort of public interaction within the
114+
community.
115+
116+
## Attribution
117+
118+
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
119+
version 2.1, available at
120+
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
121+
122+
Community Impact Guidelines were inspired by
123+
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
124+
125+
For answers to common questions about this code of conduct, see the FAQ at
126+
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
127+
[https://www.contributor-covenant.org/translations][translations].
128+
129+
[homepage]: https://www.contributor-covenant.org
130+
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
131+
[Mozilla CoC]: https://github.com/mozilla/diversity
132+
[FAQ]: https://www.contributor-covenant.org/faq
133+
[translations]: https://www.contributor-covenant.org/translations
134+

‎LICENSE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License Copyright (c) 2024 Twilio Inc.
2+
3+
Permission is hereby granted, free of
4+
charge, to any person obtaining a copy of this software and associated
5+
documentation files (the "Software"), to deal in the Software without
6+
restriction, including without limitation the rights to use, copy, modify, merge,
7+
publish, distribute, sublicense, and/or sell copies of the Software, and to
8+
permit persons to whom the Software is furnished to do so, subject to the
9+
following conditions:
10+
11+
The above copyright notice and this permission notice
12+
(including the next paragraph) shall be included in all copies or substantial
13+
portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
16+
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
17+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
18+
EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
19+
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
20+
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21+
THE SOFTWARE.

‎index.js

+178
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
import Fastify from 'fastify';
2+
import WebSocket from 'ws';
3+
import fs from 'fs';
4+
import dotenv from 'dotenv';
5+
import fastifyFormBody from '@fastify/formbody';
6+
import fastifyWs from '@fastify/websocket';
7+
8+
// Load environment variables from .env file
9+
dotenv.config();
10+
11+
// Retrieve the OpenAI API key from environment variables. You must have OpenAI Realtime API access.
12+
const { OPENAI_API_KEY } = process.env;
13+
14+
if (!OPENAI_API_KEY) {
15+
console.error('Missing OpenAI API key. Please set it in the .env file.');
16+
process.exit(1);
17+
}
18+
19+
// Initialize Fastify
20+
const fastify = Fastify();
21+
fastify.register(fastifyFormBody);
22+
fastify.register(fastifyWs);
23+
24+
// Constants
25+
const SYSTEM_MESSAGE = 'You are a helpful and bubbly AI assistant who loves to chat about anything the user is interested about and is prepared to offer them facts. You have a penchant for dad jokes, owl jokes, and rickrolling – subtly. Always stay positive, but work in a joke when appropriate.';
26+
const VOICE = 'alloy';
27+
const PORT = process.env.PORT || 5050; // Allow dynamic port assignment
28+
29+
// List of Event Types to log to the console. See OpenAI Realtime API Documentation. (session.updated is handled separately.)
30+
const LOG_EVENT_TYPES = [
31+
'response.content.done',
32+
'rate_limits.updated',
33+
'response.done',
34+
'input_audio_buffer.committed',
35+
'input_audio_buffer.speech_stopped',
36+
'input_audio_buffer.speech_started',
37+
'session.created'
38+
];
39+
40+
// Root Route
41+
fastify.get('/', async (request, reply) => {
42+
reply.send({ message: 'Twilio Media Stream Server is running!' });
43+
});
44+
45+
// Route for Twilio to handle incoming and outgoing calls
46+
// <Say> punctuation to improve text-to-speech translation
47+
fastify.all('/incoming-call', async (request, reply) => {
48+
const twimlResponse = `<?xml version="1.0" encoding="UTF-8"?>
49+
<Response>
50+
<Say>Please wait while we connect your call to the A. I. voice assistant, powered by Twilio and the Open-A.I. Realtime API</Say>
51+
<Pause length="1"/>
52+
<Say>O.K. you can start talking!</Say>
53+
<Connect>
54+
<Stream url="wss://${request.headers.host}/media-stream" />
55+
</Connect>
56+
</Response>`;
57+
58+
reply.type('text/xml').send(twimlResponse);
59+
});
60+
61+
// WebSocket route for media-stream
62+
fastify.register(async (fastify) => {
63+
fastify.get('/media-stream', { websocket: true }, (connection, req) => {
64+
console.log('Client connected');
65+
66+
67+
const openAiWs = new WebSocket('wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2024-10-01', {
68+
headers: {
69+
Authorization: `Bearer ${OPENAI_API_KEY}`,
70+
"OpenAI-Beta": "realtime=v1"
71+
}
72+
});
73+
74+
let streamSid = null;
75+
76+
const sendSessionUpdate = () => {
77+
const sessionUpdate = {
78+
type: 'session.update',
79+
session: {
80+
turn_detection: { type: 'server_vad' },
81+
input_audio_format: 'g711_ulaw',
82+
output_audio_format: 'g711_ulaw',
83+
voice: VOICE,
84+
instructions: SYSTEM_MESSAGE,
85+
modalities: ["text", "audio"],
86+
temperature: 0.8,
87+
}
88+
};
89+
90+
console.log('Sending session update:', JSON.stringify(sessionUpdate));
91+
openAiWs.send(JSON.stringify(sessionUpdate));
92+
};
93+
94+
// Open event for OpenAI WebSocket
95+
openAiWs.on('open', () => {
96+
console.log('Connected to the OpenAI Realtime API');
97+
setTimeout(sendSessionUpdate, 250); // Ensure connection stability, send after .25 seconds
98+
});
99+
100+
// Listen for messages from the OpenAI WebSocket (and send to Twilio if necessary)
101+
openAiWs.on('message', (data) => {
102+
try {
103+
const response = JSON.parse(data);
104+
105+
if (LOG_EVENT_TYPES.includes(response.type)) {
106+
console.log(`Received event: ${response.type}`, response);
107+
}
108+
109+
if (response.type === 'session.updated') {
110+
console.log('Session updated successfully:', response);
111+
}
112+
113+
if (response.type === 'response.audio.delta' && response.delta) {
114+
const audioDelta = {
115+
event: 'media',
116+
streamSid: streamSid,
117+
media: { payload: Buffer.from(response.delta, 'base64').toString('base64') }
118+
};
119+
connection.send(JSON.stringify(audioDelta));
120+
}
121+
} catch (error) {
122+
console.error('Error processing OpenAI message:', error, 'Raw message:', data);
123+
}
124+
});
125+
126+
// Handle incoming messages from Twilio
127+
connection.on('message', (message) => {
128+
try {
129+
const data = JSON.parse(message);
130+
131+
switch (data.event) {
132+
case 'media':
133+
if (openAiWs.readyState === WebSocket.OPEN) {
134+
const audioAppend = {
135+
type: 'input_audio_buffer.append',
136+
audio: data.media.payload
137+
};
138+
139+
openAiWs.send(JSON.stringify(audioAppend));
140+
}
141+
break;
142+
case 'start':
143+
streamSid = data.start.streamSid;
144+
console.log('Incoming stream has started', streamSid);
145+
break;
146+
default:
147+
console.log('Received non-media event:', data.event);
148+
break;
149+
}
150+
} catch (error) {
151+
console.error('Error parsing message:', error, 'Message:', message);
152+
}
153+
});
154+
155+
// Handle connection close
156+
connection.on('close', () => {
157+
if (openAiWs.readyState === WebSocket.OPEN) openAiWs.close();
158+
console.log('Client disconnected.');
159+
});
160+
161+
// Handle WebSocket close and errors
162+
openAiWs.on('close', () => {
163+
console.log('Disconnected from the OpenAI Realtime API');
164+
});
165+
166+
openAiWs.on('error', (error) => {
167+
console.error('Error in the OpenAI WebSocket:', error);
168+
});
169+
});
170+
});
171+
172+
fastify.listen({ port: PORT }, (err) => {
173+
if (err) {
174+
console.error(err);
175+
process.exit(1);
176+
}
177+
console.log(`Server is listening on port ${PORT}`);
178+
});

‎package.json

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"name": "twilio-speech-assistant-openai-realtime-api-node",
3+
"version": "1.0.0",
4+
"description": "",
5+
"main": "index.js",
6+
"scripts": {
7+
"test": "echo \"Error: no test specified\" && exit 1"
8+
},
9+
"keywords": [],
10+
"author": "",
11+
"license": "ISC",
12+
"type": "module",
13+
"dependencies": {
14+
"@fastify/formbody": "^8.0.0",
15+
"@fastify/websocket": "^11.0.0",
16+
"dotenv": "^16.4.5",
17+
"fastify": "^5.0.0",
18+
"ws": "^8.18.0"
19+
}
20+
}

0 commit comments

Comments
 (0)