ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository

Question

ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository

Sandeepgsn opened this issue 8 years ago · 28 comments

Sandeepgsn commented 8 years ago

While executing below command:
jruby -S gem install --local logstash-input-dynamodb-1.0.0-java.gem

Getting error:
ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository

Answer 1 · 2016-07-11T11:49:07.000Z

@Sandeepgsn try using the GEM in rubygems https://rubygems.org/gems/logstash-input-dynamodb

Answer 2 · 2016-07-21T14:59:15.000Z

@marcosnils I changed the version in the gemspec file from s.add_runtime_dependency "logstash-core", '>= 1.4.0', '< 2.0.0' to s.add_runtime_dependency "logstash-core", '>= 1.4.0', '< 2.3.4' and it worked. Do you see any problems with this?

Answer 3 · 2016-07-21T15:06:31.000Z

@sgarg-symphy I'm not the owner of this repo, I just maintain forked versions of this library because seems like AWS dropped support for this. I've pushed the gems in ruby gems and the ones there work for me.

Answer 4 · 2016-07-28T09:59:33.000Z

Struggling with the same issue.
Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository.
How to get it to work?

@marcosnils I have tried using the gems from https://rubygems.org/gems/logstash-input-dynamodb. That didn't work either. If possible can you please list out the steps.

Thank you

Answer 5 · 2016-07-28T10:36:23.000Z

@vamcyleonardo how are you installing the gem? and in which logstash version?

Answer 6 · 2016-07-28T12:17:51.000Z

@vamcyleonardo just updated the logstash-input-dynamodb gem in rubygems. Try now.

Answer 7 · 2016-07-28T13:29:07.000Z

Thank you for your response. I will try it.

Answer 8 · 2016-07-28T14:17:45.000Z

@vamcyleonardo don't clone that repo, install the gem from rubygems. Which logstash version are you using?

Answer 9 · 2016-07-28T18:07:25.000Z

@vamcyleonardo use the gem from ruby gems. Don't clone and build from this repo.

Answer 10 · 2016-07-28T18:13:34.000Z

gem install logstash-input-dynamodb:'> 2'
plugin install logstash-input-dynamodb

Answer 11 · 2016-07-28T18:26:03.000Z

gem is there https://rubygems.org/gems/logstash-input-dynamodb. Try removing the :'> 2' from the end

Answer 12 · 2016-07-28T18:38:05.000Z

maybe, give it a try

Answer 13 · 2016-07-28T18:47:50.000Z

@vamcyleonardo it should work, that's how I build my logstash docker images. Maybe if you try a clean installation it works.

If you don't want to deal with all this 💩 you can just use our docker image https://hub.docker.com/r/mantika/logstash-dynamodb-streams/

Answer 14 · 2016-07-28T19:00:29.000Z

U sure the credentials you're specifying have the correct access?

These are the policies you need

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "pointerTable0",
            "Effect": "Allow",
            "Action": [
                "dynamodb:BatchGetItem",
                "dynamodb:BatchWriteItem",
                "dynamodb:DeleteItem",
                "dynamodb:GetItem",
                "dynamodb:PutItem",
                "dynamodb:CreateTable",
                "dynamodb:Query",
                "dynamodb:Scan",
                "dynamodb:UpdateItem",
                "dynamodb:DescribeStream",
                "dynamodb:DescribeTable",
                "dynamodb:GetRecords",
                "dynamodb:GetShardIterator",
                "dynamodb:ListStreams"
            ],
            "Resource": [
                "<your arn>:table/*cptr"
            ]
        },{
            "Sid": "streamReading1",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeStream",
                "dynamodb:DescribeTable",
                "dynamodb:Scan",
                "dynamodb:ListStreams",
                "dynamodb:GetShardIterator",
                "dynamodb:GetRecords"
            ],
            "Resource": [
                "<table ARN>",
                "<stream ARN>"
            ]
        }
    ]
}

Answer 15 · 2016-07-28T19:03:47.000Z

Remember that you also need to enable the table stream in dynamoDB

Answer 16 · 2016-08-24T22:22:18.000Z

When running the docker image, i get "Couldn't find any input plugin named 'dynamodb'". Any idea why?

Answer 17 · 2016-08-24T22:56:14.000Z

Which image are you running? V1 or v2?

sent from mobile

On Aug 24, 2016 7:22 PM, "Mike Colarusso" notifications@github.com wrote:

When running the docker image, i get "Couldn't find any input plugin named
'dynamodb'". Any idea why?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#21 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABgV2gm5_gQS-C6loNbtCBj9WIZAvHd0ks5qjMQagaJpZM4JH5q2
.

Answer 18 · 2016-08-24T22:59:56.000Z

I tried latest and 2

From: Marcos Nils
Reply-To: awslabs/logstash-input-dynamodb
Date: Wednesday, August 24, 2016 at 6:56 PM
To: awslabs/logstash-input-dynamodb
Cc: Michael Colarusso, Comment
Subject: Re: [awslabs/logstash-input-dynamodb] ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository (#21)

Which image are you running? V1 or v2?

sent from mobile

On Aug 24, 2016 7:22 PM, "Mike Colarusso" <notifications@github.com mailto:notifications@github.com> wrote:

When running the docker image, i get "Couldn't find any input plugin named
'dynamodb'". Any idea why?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#21 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABgV2gm5_gQS-C6loNbtCBj9WIZAvHd0ks5qjMQagaJpZM4JH5q2
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/21#issuecomment-242233947, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFVq4jXMtza36zX3kN9XvYtQD4sf7J8Fks5qjMwRgaJpZM4JH5q2.

Answer 19 · 2016-08-24T23:45:34.000Z

@mcolarusso I've just updated the images, I believe those were outdated. Can you please update and try again?

Answer 20 · 2016-08-25T14:14:43.000Z

@marcosnils That seems to have fixed the issue.

Answer 21 · 2016-08-31T21:39:58.000Z

What is this command "bin/plugin" ?
I don't see this anywhere, even after installing "gem install ..."

Answer 22 · 2016-09-08T22:30:51.000Z

Hello, I tried using the mantika's Docker image and I had this problem:

Pipeline aborted due to error", :exception=>com.amazonaws.AmazonClientException: Unable to execute HTTP request: dynamodb.us-east-1.amazonaws.com, :backtrace=>["com.amazonaws.http.AmazonHttpClient.executeHelper(com/amazonaws/http/AmazonHttpClient.java:618)"

Any idea what's going on here?
Thanks!!

Answer 23 · 2016-09-09T04:58:27.000Z

@feedtherich can u share your config?

Answer 24 · 2016-09-09T17:54:00.000Z

maybe i'm doing this wrong. i'm new to docker. i am just running this script:

sudo docker run mantika/logstash-dynamodb-streams -e '
input { dynamodb {
        endpoint => "dynamodb.us-east-1.amazonaws.com"
        streams_endpoint => "streams.dynamodb.us-east-1.amazonaws.com"
        view_type => "new_and_old_images"
        aws_access_key_id => "<redacted for security>"
        aws_secret_access_key => "<redacted for security>"
        table_name => "<redacted for security>"
}   }
filter { dynamodb {} }
output { elasticsearch { host => localhost } stdout { } } '

Answer 25 · 2016-09-10T22:43:39.000Z

@feedtherich if you're using the "latest" image you need to setup the output plugin as follows:

output {
elasticsearch {
hosts => ["elasticsearch"]
}
}

note that hosts is pluran and an array of values.

Answer 26 · 2016-09-12T21:56:08.000Z

Thanks.
I still got the same error, however.
I think the thing I'm doing wrong is assuming everything is happening
automatically without me needing logstash setup locally.
Or, am I supposed to do something with one of the two Docker files?
I have no idea how to use them.

Thanks.

On Sat, Sep 10, 2016 at 3:43 PM, Marcos Nils notifications@github.com
wrote:

@feedtherich https://github.com/feedtherich if you're using the
"latest" image you need to setup the output plugin as follows:

output {
elasticsearch {
hosts => ["elasticsearch"]
}
}

note that hosts is pluran and an array of values.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#21 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AUYcgolTZ-5N5c9zrZzRnFvKkkCCbbzgks5qozKdgaJpZM4JH5q2
.

Answer 27 · 2016-10-17T05:55:08.000Z

I am unable to install this plugin on my EC2 machine(Amazon linux). when I try this command:
jruby -S gem install --local logstash-input-dynamodb-1.0.0-java.gem
I am facing this error:
ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository
Please help

Answer 28 · 2019-10-17T20:37:10.000Z

I may be wrong, but
I think the issues are due to postfix "-java" the gems have in the repository. https://rubygems.org/downloads/logstash-core-2.3.1-java.gem.

All other packets look like https://rubygems.org/downloads/dynamodb-1.2.1.gem
i.g. nothing goes after version, only .gem.