Does not seem to pull latest pushes from bitbucket repo.
gyalbu opened this issue · 14 comments
First of all, thanks for sharing this. Script looks great and I was testing the script out with a tiny example repo on bitbucket. The initial clone was successful. I mean - on the first run, it creates a "git" folder in my server and puts the repository in there. It also successfully syncs the project files into my production-folder. But since then the script seems to be stuck on the same commit, while there has been many more commits after the initial commit.
I have checked bitbucket webhook and all is fine there. Here is a link to a gist of the result when the deploy.php is run directly. Please let me know if you need more details. Will be great to get this sorted. Thanks.
Hello @gyalbu - thanks for trying this script.
My suggestion as first step to troubleshoot is to check the webhook log in Bitbucket. Click on View Requests. You should see a list of each time the webhook was triggered, including the date-time, time it took to complete, and HTTP response code.
If you don't see anything there it means that the webhook is not being triggered at all. You need to check the webhook configuration and make sure the Triggers are set to the Repository push option.
If there are items in the requests list, are they marked with an error label? Or do they have the green 200 label? In either case click on View Details, and let me know what you found there. If there was an error during execution of the script it should appear there.
If there are no error reported there, I suggest you configure the deploy script to write to a log file so that you can see the complete execution.
Let me know what you find.
@lyquix-owner I appreciate the quick response. I think deploy.php itself returns a pretty verbose report with all the echo statements at almost every step. Quite similar to what a log would be. Issue is, I am new to git and unable to figure out what git commands are doing at each step.
I can assure you that my SSH authentication/connection to bitbucket is fine. Also, I have been checking the response body, returned from the web-server. It is similar to the gist I linked to earlier ( except the manual trigger part ).
There is something that I have noticed. When I am in my servers folders and files screen, I noticed that when deploy.php is pinged ( from bitbucket on push ) there is an immediate change in the modified-time of deploy.php and version file. I also noticed this change in the corresponding / local .git folder in server ( outside the web-directory ). This is how I know that there IS a communication between bitbucket repo, deploy.php and .git folder. But there are no change in the modified-time of test-files as expected.
I also dont think the dependency library "rsync" is at any kind of fault here because it may not be able to do anything if there is no change in files in the local git folder itself.
As I said the script gets stuck in the last commit on every subsequent pings after the initial run. I added a var_dump($commits) around line number 341 and it keeps showing the same last commit.
TBH, I think it should work in my environment because there is another script , which has a similar workflow, that works pretty well in the same environment. It also uses a local git on server and then deploys the changes in the production folder.
Here is some info on my test environment. The test files are at a shared hosting account at hostgator. Git and rsync seems to be working fine.
<b>/usr/bin/git</b> : git version 1.7.9.5
<b>/usr/bin/rsync</b> : rsync version 3.0.6 protocol version 30
Just in case - Here is a link to my config file.
Here is a link to the response body returned by deploy request.
Here are some bottom-lines. Has this script been tested in an environment similar to mine ? Is it possible that my environment may be limiting some feature in your script ?
I really wish to get this working with your help, as your script offers some great additional features, I would like to customize and use. Please let me know if you need any more information. Thanks.
@gyalbu - do you have the VERSION file tracked in the repo?
@lyquix-owner If you mean, do I have the VERSION file in the local repo folder in my web-server then, the answer is No. But I have it in my web-directory where I believe it was automatically added by the script.
I have kept things minimal as possible in my test.
Here is a screenshot of the local repo on my webserver.
And here is a screenshot of test web directory.
Notice how the modified-time is same for deploy folder, and VERSION file in the web-directory. Also it is the same for .git folder. But it does not change at all for the actual test-file.
EDIT - I was testing around a bit more and found something interesting, If I delete the test file from my server repo and then make a new push at bitbucket, then the latest testfile is pulled from bitbucket to the server-repo. Following that I think rsync kicks in and replaces the testfile at my web-directory too.
This leads me to believe that there should be no testfiles/projectfiles in the server-repo folder except the .git folder. But, most probably I am wrong and your script operates with a different logic. Let me know your suggestions please. Thnx.
Bump.
Hi @gyalbu - here are my thoughts:
- It is expected for the deploy folder to be updated because you have the log file in there, which will be written on every execution of the script.
- The same for the VERSION file that is written on every execution
- It seems like the directory is already in the most recent version, so there is nothing to change. In the log you sent I can read the following:
HEAD is now at 2e0ca8c
Current target directory version is 2e0ca8c
rsync doesn't work as a regular cp command - if the file in your target directory is the same as the one in the git directory, it is expected that there will be no change/update to the file.
I suggest you make a change to lorem.html in Bitbucket, and check if the file is updated in the target directory.
@lyquix-owner I think I have mentioned this earlier. But I have been making changes to lorem.html and there has been many commits after 2e0ca8c but the file at the server is simply not getting any updates. Neither in the git directory nor in target directory. I have told you almost everything about my environment. Please let me know if you have any further suggestions.
From the webhook response it seems like in your git directory the most recent commit was 2e0ca8c, after fetching from the repo.
Have the more recent commits made on the 'master' branch?
If you delete files lorem.html and VERSION from the target directory, and make a new commit in Bitbucket (in the master branch), what is the result?
Yes, multiple test commits have been made to "master" branch after the mentioned commit. Also, master branch is the only branch in this testrepo.
As instructed, I deleted the lorem.html and VERSION files from git-directory as well as target-directory. Then I made another commit - 28f2434. It is the most recent commit, as you can see in the screenshot below. This way my test file lorem.html got updated on both git and target directories. But a weird thing is it DID NOT get updated to my recent commit - 28f2434, but another one from yesterday - e09f55e. I have marked the position of these relevant commits in the screenshot below. You can see the time these commits occurred as well.
This event leads me to think, is the script somehow getting affected by the server's timezone. Is timezone somehow a factor in the script ? Is it possible that a time mismatch between the server and remote repo is, interfering with the updates somehow ? Also, have you tested the script yourself recently. Is it possible that bitbucket may have made some changes in their payload structure and the script is not optimized to adapt to that yet. Any other clues on possible culprits ?
Very strange. We have 55 repos using this script and we use it daily. Never had a problem like this. Time zone has no bearing in the functioning of git or the script.
Can you share the log file?
Just found out that my shared hosting account server is running on CentOS 6.8 Enterprise Linux x86.
You mentioned in the readme - These instructions were created for Ubuntu 14.04 and tested also on Ubuntu 16.04. You should be able to adjust them to other Linux distros. So is there a remote chance that the script may not work as expected in CentOS ?
I performed another test using manual trigger. The commit parameter "c", is set to my most recent commit on BB. Url - http://xxxxxxxxxx.com/deploy/deploy.php?t=secret-lorem&b=master&c=28f2434. Again, no updates to lorem.html.
Please take a look at the full and most recent log here, for the recent ping from BB. I realized I have not sent you a full log earlier.
My config.php remains the same. Also I have not made any changes to deploy.php, only except changing deploy-config.php to config.php. Since BB repo is successfully communicating with the repo-directory and the target directory, I dont know where else to look for errors.
Hi @gyalbu - I think if CentOS were a problem, we would see an error in the execution, or problems setting up the keys, but there is none.
What I am seeing from the log is that after fetching from origin (BitBucket) your local repo seems to have commit e09f55e as the most recent. That's clearly wrong. My only guess at this point is that your server is running a pretty old version of git (rsync is pretty old too) from 2010. This script was developed less than a year ago and we have not tested it with old versions of git. I suggest you update both.
I also suggest you go to the command line and execute manually the git same git commands you see in the script. Note that if you are in the git directory you don't need the parameters --git-dir and --work-tree, for example:
git fetch --tags origin master
git checkout master
git --no-pager log --pretty=format:"%h" origin/master
The last command should produce the complete list of ALL commits in the repo.
I hope this helps, let me know what you find.
@lyquix-owner I am in a git-directory when I ran the suggested commands and here is the result of it all.
So, first I ran - git remote show origin to check if the git is referring to correct repository.
lorem@test.com [~/secondtest]# git remote show origin
* remote origin
Fetch URL: xxxxx@bitbucket.org:xxxxx/secondtest.git
Push URL: xxxxx@bitbucket.org:xxxxx/secondtest.git
HEAD branch: master
Remote branch:
master tracked
Local branch configured for 'git pull':
master merges with remote master
Local ref configured for 'git push':
master pushes to master (local out of date)
Then I ran the suggested commands.
lorem@test.com [~/secondtest]# git fetch --tags origin master
From bitbucket.org:xxxxx/secondtest
* branch master -> FETCH_HEAD
lorem@test.com [~/secondtest]# git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
lorem@test.com [~/secondtest]# git --no-pager log --pretty=format:"%h%" origin/master
4c73e75%
lorem@test.com [~/secondtest]# lorem@test.com [~/secondtest]# git fetch --tags origin master
From bitbucket.org:xxxxx/secondtest
* branch master -> FETCH_HEAD
lorem@test.com [~/secondtest]# git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
lorem@test.com [~/secondtest]# git --no-pager log --pretty=format:"%h" origin/master
4c73e75
9200731
Since I have been continuously making various test commits, currently "4c73e75" is the most recent commit in the BB repo, which is yet to be pulled to the webserver. "9200731" is the second last which IS on the server. Please let me know if these results make any sense to you.
In the meanwhile I will perform a test to determine if rsync can do what we expect it to.
My guess is that the version of git is too old and the commands don't work as expected.