Unable to connect to MQTT Broker

Question

Unable to connect to MQTT Broker

Closed this issue 11 years ago · 40 comments

As requested on Twitter:

I've several Mosquitto MQTT brokers running at hone.

The one I'd like to connect to is on 192.168.8.2.1884 and requires a username and password. When adding a cloud service however, I cannot specify other ports as the protocol prefixes default to 1883 or 8883.

Additionally, when I try to connect to my local broker on 1883 that does not require username and password I cannot as the interface forces me to enter the credentials before connecting.

When I try to connect to 1883 with bogus credentials, "INVALID DEVICE /DEVICE/INDICATOR/TEXT/MQTT" is thrown in the interface as an error. The same is thrown if I connect to an SSL and username/password protected broker by specifying the mqtts protocol prefix.

Could you add anonymous auth and the possibility to specify an other port? Besides that, how do I connect the steward to my SSL protected broker on port 8883?

binarybucks commented 11 years ago

Huh?

Answer 1 · 2014-02-18T18:17:19.000Z

thanks for the note.

issue #1: you want to use a URL that looks like this:

    mqtt://hostname:portno/path?options

or

    mqtts://hostname:portno/path?options

if portno isn't specified you get 8883 (mqtts) or 1883 (mqtt), respectively.

path is what gets published to

options is expressed as the usual ?k1=v1&k2=v2&… where your choices are:

    keepalive=60
    reconnectPeriod: 1000
    clean: true
    encoding: utf8

personally, i would have preferred that the MQTT gurus (hint, hint) defined an MQTT URI scheme, but since they didn't, this is what i'm using…

issue #2: one of the client files was out of date and asking for the wrong device path. i fixed that, but you'll need to grab the latest from github.

ps: if you put up a public broker somewhere, i'd be happy to configure my steward to push some measurements to it!

Answer 2 · 2014-02-18T18:54:06.000Z

I'll give it a try.
Once thing. When entering a bad username/password for the broker and saving it, the steward crashes, restarts, tries to connect again, crashes, etc.

(without format as it gets eaten by GH)

alert: [steward] exception diagnostic=Connection refused: Bad username or password
alert: [steward] exception stack=[{"fileName":"/home/alexander/thethingsystem/steward/steward/node_modules/mqtt/lib/client.js","lineNumber":474,"functionName":"MqttClient._handleConnack","typeName":"MqttClient","methodName":"_handleConnack","columnNumber":9,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/node_modules/mqtt/lib/client.js","lineNumber":194,"functionName":"","typeName":"Connection","methodName":null,"columnNumber":10,"native":false},{"fileName":"events.js","lineNumber":95,"functionName":"Connection.EventEmitter.emit","typeName":"Connection","methodName":"EventEmitter.emit","columnNumber":17,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/node_modules/mqtt/lib/connection.js","lineNumber":187,"functionName":"Connection._write","typeName":"Connection","methodName":"_write","columnNumber":12,"native":false},{"fileName":"_stream_writable.js","lineNumber":223,"functionName":"doWrite","typeName":"Object","methodName":null,"columnNumber":10,"native":false},{"fileName":"_stream_writable.js","lineNumber":213,"functionName":"writeOrBuffer","typeName":"Object","methodName":null,"columnNumber":5,"native":false},{"fileName":"_stream_writable.js","lineNumber":180,"functionName":"Connection.Writable.write","typeName":"Connection","methodName":"Writable.write","columnNumber":11,"native":false},{"fileName":"_stream_readable.js","lineNumber":583,"functionName":"write","typeName":"Object","methodName":null,"columnNumber":24,"native":false},{"fileName":"_stream_readable.js","lineNumber":592,"functionName":"flow","typeName":"Object","methodName":null,"columnNumber":7,"native":false},{"fileName":"_stream_readable.js","lineNumber":624,"functionName":"Socket.pipeOnReadable","typeName":"Socket","methodName":"pipeOnReadable","columnNumber":5,"native":false}]
uncaught exception: Error: Connection refused: Bad username or password
Press ^C to exit or wait 10 seconds to restart

Answer 3 · 2014-02-18T18:54:49.000Z

yikes. i guess i'd better put a catch around that… thanks for repotting it.

Answer 4 · 2014-02-18T19:04:00.000Z

and thats now fixed!

Answer 5 · 2014-02-18T19:05:25.000Z

Now I'm getting this right after starting the steward (already cleaned the db)

info: [wearable] loading watch driver
alert: [steward] exception diagnostic=Cannot read property '0' of undefined
alert: [steward] exception stack=[{"fileName":"/home/alexander/thethingsystem/steward/steward/actors/actor-place.js","lineNumber":440,"functionName":"Place.getWoeID","typeName":"Place","methodName":"getWoeID","columnNumber":32,"native":false},{"fileName":"exports.Place (/home/alexander/thethingsystem/steward/steward/actors/actor-place.js","lineNumber":168,"functionName":"new","typeName":"Object","methodName":null,"columnNumber":30,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/actors/actor-place.js","lineNumber":773,"functionName":"","typeName":"Statement","methodName":null,"columnNumber":9,"native":false}]
uncaught exception: TypeError: Cannot read property '0' of undefined
Press ^C to exit or wait 10 seconds to restart

Answer 6 · 2014-02-18T19:12:01.000Z

sorry about that, the weather code added last week missed a robustness check. i've added it.

Answer 7 · 2014-02-18T19:27:22.000Z

Great, I'm now able to connect as desired.
While we're at it, can you hint me how to configure the Steward that it picks up my presence via OwnTracks?

Answer 8 · 2014-02-18T21:07:40.000Z

i'm glad we finally finished the horizon effect on that set of bugs… (-;

ok, for owltracks:

the first thing is that we need to get your steward to the internet via a taas cloud. you can run your own, or you can use mine (you'd be the third user on there). if you want to run your own, you want to look at:

    https://github.com/TheThingSystem/taas-server

which requires that you have a VPS account somewhere. if you want to go this route, i can help you, although it's going to take a few round-trips.

if you just want a TAAS account, this is what i need from you:

the "name" of your steward. this is what you sent when you click on the gearbox on the default client. the default is "home" and you need to pick something else.
the uuid of your steward, when your steward starts, one line of console output looks like this:
```
notice: [steward] start uuid=2f402f80-da50-11e1-9b23-XXXXXXXXXX
```

i need to know the entire value.

what i will do is take that information and create an account on my TAAS server and send you back a file that you put in steward/steward/db/ along with the self-signed CRT file for the TAAS cloud.
when you restart the steward, it will automatically connect to the cloud and awaiting for incoming connections, e.g., from own tracks.

ok, let's assume that all that is done, one way or another, here are my preferences for owntracks on android:

HOST: IP address of cloud server
Port: 8883
Client ID: STEWARDNAME/PHONENAME
Username: owntracks/STEWARDNAME/PHONENAME
Password: ….
Security: TLS (Custom Certificate)
CA.crt path: /storage/emulated/0/Downloads/cloud.crt

the topic is the same as the username

Answer 9 · 2014-02-18T21:15:13.000Z

Mhh, but I have my OwnTracks broker and the Steward running on the same host while both are connected perfectly well. Also, the broker is reachable from the internet from my OwnTracks application, so there's theoretically no need for any NAT bypassing that requires me to connect the steward to any $cloud?

Can't the steward just connect to my local broker and subscribe to the topic of OwnTracks to automagically discover me?

Answer 10 · 2014-02-18T21:16:25.000Z

good point. most people don't have their own broker, but we ought to support that. let me do some research. back in a bit.

Answer 11 · 2014-02-18T21:18:20.000Z

Well, I know quite some people with their own broker, but that probably depends on my circle of acquaintances :D
Go ahead and take your time.

Answer 12 · 2014-02-18T23:02:56.000Z

ok, get the latest from github.

if you have a /device/indicator/mqtt/text active, it should accept messages too.

Answer 13 · 2014-02-19T03:45:31.000Z

by the way: if you start a service (preferably commercial) that offers a broker for owntracks users or users of jpmens' mqtt warn, that'd be great! i'd love to have the steward talking to those

Answer 14 · 2014-02-20T18:14:20.000Z

@binarybucks - where did we end up on this? thanks!

Answer 15 · 2014-02-20T18:20:52.000Z

sorry for the confusion. what i wanted to know is whether you had gotten the latest from github and if so, whether you were getting bi-directional traffic to your /device/indicator/mqtt/text device, and whether the incoming traffic from owntracks was working to populate a /device/presence/owntracks/mobile device ...

Answer 16 · 2014-02-20T18:46:23.000Z

I'd test it but with the latest code the steward dies on me with:

notice: [steward] start uuid=2f402f80-da50-11e1-9b23-00252289e614
alert: [steward] exception diagnostic=dns service error: name conflict
alert: [steward] exception stack=[{"fileName":"Advertisement (/home/alexander/thethingsystem/steward/steward/node_modules/mdns/lib/advertisement.js","lineNumber":56,"functionName":"new","typeName":"Object","methodName":null,"columnNumber":10,"native":false},{"fileName":"[as createAdvertisement] (/home/alexander/thethingsystem/steward/steward/node_modules/mdns/lib/advertisement.js","lineNumber":64,"functionName":"Object.create","typeName":"Object","methodName":"create","columnNumber":10,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/core/server.js","lineNumber":321,"functionName":"exports.advertise","typeName":"exports","methodName":"advertise","columnNumber":17,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/core/server.js","lineNumber":260,"functionName":null,"typeName":null,"methodName":null,"columnNumber":5,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/node_modules/portfinder/lib/portfinder.js","lineNumber":48,"functionName":"Server.onListen","typeName":"Server","methodName":"onListen","columnNumber":5,"native":false},{"fileName":"events.js","lineNumber":180,"functionName":"Server.g","typeName":"Server","methodName":"g","columnNumber":16,"native":false},{"fileName":"events.js","lineNumber":92,"functionName":"Server.EventEmitter.emit","typeName":"Server","methodName":"EventEmitter.emit","columnNumber":17,"native":false},{"fileName":"net.js","lineNumber":1052,"functionName":null,"typeName":null,"methodName":null,"columnNumber":10,"native":false},{"fileName":"node.js","lineNumber":415,"functionName":"process._tickCallback","typeName":"process","methodName":"_tickCallback","columnNumber":13,"native":false}]
uncaught exception: Error: dns service error: name conflict

Answer 17 · 2014-02-20T19:18:30.000Z

Any chance a steward is running somewhere else on the LAN or server?

/mtr

On Feb 20, 2014, at 10:46, Alexander Rust notifications@github.com wrote:

I'd test it but with the latest code the steward dies on me with:

notice: [steward] start uuid=2f402f80-da50-11e1-9b23-00252289e614
alert: [steward] exception diagnostic=dns service error: name conflict
alert: [steward] exception stack=[{"fileName":"Advertisement (/home/alexander/thethingsystem/steward/steward/node_modules/mdns/lib/advertisement.js","lineNumber":56,"functionName":"new","typeName":"Object","methodName":null,"columnNumber":10,"native":false},{"fileName":"[as createAdvertisement] (/home/alexander/thethingsystem/steward/steward/node_modules/mdns/lib/advertisement.js","lineNumber":64,"functionName":"Object.create","typeName":"Object","methodName":"create","columnNumber":10,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/core/server.js","lineNumber":321,"functionName":"exports.advertise","typeName":"exports","methodName":"advertise","columnNumber":17,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/core/server.js","lineNumber":260,"functionName":null,"typeName":null,"methodName":null,"columnNumber":5,"native":false},{"fileName":"/home/alexander/thethingsystem/steward/steward/node_modules/portfinder/lib/portfinder.js", "lineNumber":48,"functionName":"Server.onListen","typeName":"Server","methodName":"onListen","columnNumber":5,"native":false},{"fileName":"events.js","lineNumber":180,"functionName":"Server.g","typeName":"Server","methodName":"g","columnNumber":16,"native":false},{"fileName":"events.js","lineNumber":92,"functionName":"Server.EventEmitter.emit","typeName":"Server","methodName":"EventEmitter.emit","columnNumber":17,"native":false},{"fileName":"net.js","lineNumber":1052,"functionName":null,"typeName":null,"methodName":null,"columnNumber":10,"native":false},{"fileName":"node.js","lineNumber":415,"functionName":"process.tickCallback","typeName":"process","methodName":"tickCallback","columnNumber":13,"native":false}]
uncaught exception: Error: dns service error: name conflict

—
Reply to this email directly or view it on GitHub.

Answer 18 · 2014-02-20T20:02:00.000Z

Ah you were right, there was a stray tmux session still running the steward.
However, I'm not getting any updates to the owntracks device.
I specified the broker connection as an MQTT cloud service, did I do this right or miss anything?

Answer 19 · 2014-02-21T11:29:01.000Z

ok, let's start the debugging process. go to:

    http://steward.local:8887/console

look for /device/indicator/mqtt/text and see if any devices are present under there. if not, you need to create it using

    http://steward.local:8887/

otherwise, see if the parameters look right to you.

Answer 20 · 2014-02-21T21:25:58.000Z

also, if you think it would be faster for me to connect to server and try things out, please give me a (temporary) URL, username, and password… thanks!

Answer 21 · 2014-02-21T21:35:57.000Z

I'm just restructuring my network setup and migrating servers. Give me a day or two ;)

Answer 22 · 2014-02-21T21:51:12.000Z

That works!

/mtr via iPhone

On Feb 21, 2014, at 13:35, Alexander Rust notifications@github.com wrote:

I'm just restructuring my network setup and migrating servers. Give me a day or two ;)

—
Reply to this email directly or view it on GitHub.

Answer 23 · 2014-02-21T21:53:49.000Z

At the moment I'm getting connection attempts from the steward on the broker but they fail on the broker side with

1393019490: OpenSSL Error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
1393019490: Socket read error on client (null), disconnecting.
1393019491: New connection from 37.4.206.202 on port 8883.

What SSL/TLS version does the steward use? My mosquitto is configured for TLSv1 to allow OwnTracks to connect.

Answer 24 · 2014-02-22T19:32:58.000Z

perhaps the real question is what SSL/TLS version does the node.js MQTT package use - https://github.com/adamvr/MQTT.js - from lib/mqtt.js

        var tls_client = tls.connect(port, host, tls_opts, function() {

in general, node doesn't really care:

    http://nodejs.org/api/tls.html#tls_tls_connect_options_callback

but it can only use what is configured for your system.

Answer 25 · 2014-02-23T13:14:45.000Z

Mhh, I setup a broker with a plaintext listener on 1883.
The problem is that I'm constantly getting a connack (4) aka bad username or password on the broker. mosquitto_sub is connecting just fine with the same credentials.

What I'm wondering is, that the console page is showing the username entry for the mqtt/1 device as **** while the password is shown in plain text. Is there something mixed up?

Answer 26 · 2014-02-24T03:24:33.000Z

i think there are two options:

option 1 is to trying using the MQTT.js package to talk the broker and see what it takes to get that working. once we know that, it should be easy for me to fix the steward to do it "that way"

option 2 is to let me try to connect to a server and debug it on this end (i'll also start by using just MQTT.js)

the reason that that console spits out "****" for the password is that certain property values are elided on output, pass phrases being one of them. if you want to see what the real value is, you need to look in the database… apparently security folks dig that "extra touch of inconvenience" (-;

thanks for your continued patience!

Answer 27 · 2014-02-24T07:54:39.000Z

if you send me a mail, I can spin up another Mosquitto instance for you and
drop you the credentials.

Re the passwords: Not the password is shown as ****, the username is. The
password is shown in the clear.

Mit freundlichen Grüßen / With kind regards
Alexander Rust
--Sent from mobile
On Feb 24, 2014 4:24 AM, "mrose17" notifications@github.com wrote:

i think there are two options:

option 1 is to trying using the MQTT.js package to talk the broker and see
what it takes to get that working. once we know that, it should be easy for
me to fix the steward to do it "that way"

option 2 is to let me try to connect to a server and debug it on this end
(i'll also start by using just MQTT.js)

the reason that that console spits out "****" for the password is that
certain property values are elided on output, pass phrases being one of
them. if you want to see what the real value is, you need to look in the
database… apparently security folks dig that "extra touch of inconvenience"
(-;

thanks for your continued patience!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/161#issuecomment-35855042
.

Answer 28 · 2014-02-24T08:06:43.000Z

here i am!

Answer 29 · 2014-02-24T08:07:20.000Z

i believe that both username and passphrase are elided. i will remove the eliding for username...

Answer 30 · 2014-02-26T01:03:56.000Z

all fixed… thank you alexander!

Answer 31 · 2014-03-08T05:55:58.000Z

Sorry to post on this closed issue but is the best resource I found until now regarding MQTT and steward - I read almost all wiki and couldn't find anything, this one clarifies things a bit. I thought the MQTT module starts an MQTT server but apparently not. So please help me understand all the steps as I would really like to figure this out.

install mosquitto on my local machine and make it available from outside my local network
configure it somehow to accept messages from OwnTracks - I'll find details on the web I assume
make steward talk to mosquitto - how exactly?
create the necessary devices (/device/indicator/mqtt/text and /device/presence/owntracks/mobile) - how? not sure how devices can be created in the d3 client, maybe I missed something in the docs

The other alternative would be to use a taas server but since I know nothing about that and would introduce one more variable, I guess I'll avoid this route.

Again, I apologize fro posting here on a closed issue, not sure where else to ask questions. Google+? Here, in issues but start a new one?

Thanks for your help and patience.

Answer 32 · 2014-03-08T12:35:42.000Z

hi. sorry for the confusion. here's what you need to do:

set up a broker. mosquito should be fine for this purpose.
get owntracks running on your mobile device and talking to your broker.
then:

for the steward, you need to create just one device, /device/indicator/mqtt/text, using the HTML5/D3 client. click on the settings gearbox in the upper-left hand corner, scroll down to cloud services and select mqtt. the username and passphrase should be obvious. the URL has three important parts"

scheme, either 'mqtt://' or 'mqtts://' depending on whether your broker is using encrypted traffic
broker, either the domain name or IP address
topic prefix, which can be blank or something like owntracks/steward

putting that together you might have

    mqtts://BROKER/owntracks/steward

then click on 'ADD CLOUD SERVICE'.

the client will automatically subscribe to owntracks/#, so when your mobile device publishes to your broker, the steward will see those entries and automatically create /device/prense/owntracks/mobile devices for you…

good luck!

Answer 33 · 2014-03-08T14:20:21.000Z

We should add this to the documentation somewhere? Where do you think it'd be appropriate?

Answer 34 · 2014-03-09T03:13:24.000Z

ideally, i'd like to see a page in the wiki for each deviceType that has a how-to list, plus a video. perhaps we can start that section for MQTT?

@aallan - if you agree, could you open an issue?

Answer 35 · 2014-03-09T16:55:02.000Z

Opened an issue on the website issue list.

Answer 36 · 2014-03-10T03:34:54.000Z

Thank you so much for all the details, this is just what i was looking for.

Just one question: I tried the steps you gave me and I never get a message
from the broker. It's true, I didn't install my own broker, just because I
didn't want to have to deal with port forwarding and stuff. So I used a
public one (broker.mqttdashboard.com) that has no security and no
username/password. Maybe my configuration didn't work because it requires
username/password. I tried to figure out from the code if this is the issue
but being a beginner with both javascript and nodejs, it's not very clear
to me which class is responsible for creating the mqtt subscriber.

Can you please tell me which class should I look at to figure out if
username/password are required?

Thanks again!

claudiu

On Sat, Mar 8, 2014 at 5:35 AM, mrose17 notifications@github.com wrote:

hi. sorry for the confusion. here's what you need to do:

set up a broker. mosquito should be fine for this purpose.

get owntracks running on your mobile device and talking to your
broker.

then:

for the steward, you need to create just one device,
/device/indicator/mqtt/text, using the HTML5/D3 client. click on the
settings gearbox in the upper-left hand corner, scroll down to cloud
services and select mqtt. the username and passphrase should be obvious.
the URL has three important parts"

scheme, either 'mqtt://' or 'mqtts://' depending on whether your
broker is using encrypted traffic

broker, either the domain name or IP address

topic prefix, which can be blank or something like owntracks/steward

putting that together you might have
mqtts://BROKER/owntracks/steward
then click on 'ADD CLOUD SERVICE'.

the client will automatically subscribe to owntracks/#, so when your
mobile device publishes to your broker, the steward will see those entries
and automatically create /device/prense/owntracks/mobile devices for you...

good luck!

Reply to this email directly or view it on GitHubhttps://github.com//issues/161#issuecomment-37096512
.

Answer 37 · 2014-03-10T06:44:45.000Z

at the present time, the username/password is required by the HTML5/D3 client.

if the broker doesn't require a username/password, try inputting something like "test" / "123456" and see if that works. if not, perhaps @binarybucks can point you at a broker than you can test with...

Answer 38 · 2014-06-04T07:11:23.000Z

I'm trying to configure mqtt. I'm using a Mac, and I have mosquitto running on the Mac on port 1883.

I had tried to set this up with test.mosquitto.org, but that site does not use user/password authentication. So I figure I need to reconfigure. But somehow the system still has test.mosquitto.org mapped to /device/indicator/mqtt/text (or mqtt device/1) and I can't figure out where to go to change the hostname it's pointing at.

Answer 39 · 2014-06-04T07:17:40.000Z

A clue, perhaps; in the log file I see this

warning: [manage] wss 127.0.0.1 62566 /manage permanent=false, diagnostic=duplicate uuid, videlicet=device/1, requestID=3, event=device creation

just after trying to update my mqtt settings.