Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Progress goes over 100% #533

Open
ManuelBerrueta opened this issue Oct 16, 2024 · 5 comments
Open

Progress goes over 100% #533

ManuelBerrueta opened this issue Oct 16, 2024 · 5 comments

Comments

@ManuelBerrueta
Copy link

Using the latest release at this time (v3.6) for Linux. I was running the tool to enumerate virtual hosts for a lab and the tool is going way over the 100% mark:
image

@firefart
Copy link
Collaborator

Can you please try the dev branch and see if the problem exists there too?

go install github.com/OJ/gobuster/v3@dev

@ManuelBerrueta
Copy link
Author

Hello, I tried to install it and got an issue with the library: /usr/libexec/gcc/x86_64-linux-gnu/13/cc1: error while loading shared libraries: /lib/x86_64-linux-gnu/libisl.so.23: invalid ELF header. I would usually explore more about the issue, but time constraints!

@NITHINSPACETIME
Copy link

Hello, I tried to install it and got an issue with the library: /usr/libexec/gcc/x86_64-linux-gnu/13/cc1: error while loading shared libraries: /lib/x86_64-linux-gnu/libisl.so.23: invalid ELF header. I would usually explore more about the issue, but time constraints!

this error occurs while the library file /lib/x86_64-linux-gnu/libisl.so.23 is corrupted or invalid i guess... To solve this error try checking if this file exists : ls -l /lib/x86_64-linux-gnu/libisl.so.23

if it does then , try insatlling : sudo apt update sudo apt install --reinstall libisl23

if anything happenes ask !!!

acammack added a commit to acammack/gobuster that referenced this issue Jan 29, 2025
Specifying patterns for the word list will no longer cause progress to
go past 100%. Additionally, the GobusterDir transformations for file
extensions and backups will be applied after pattern expansion.

Fixes OJ#405, OJ#480, and OJ#533
firefart pushed a commit that referenced this issue Feb 19, 2025
* Account for patterns in total tries

Specifying patterns for the word list will no longer cause progress to
go past 100%. Additionally, the GobusterDir transformations for file
extensions and backups will be applied after pattern expansion.

Fixes #405, #480, and #533

* Run backup detection on success only

This is done by re-arranging the code from exiting on channel close to
using the contexts and the results counters to signal the end of work
processing. A little more work is needed to prevent infinite loops
caused by devious services/misconfiguration and to expose to the cli the
ability to have patterns run on successful finds.

Fixes #298

* Prioritize stopping when the context is done

From the Go spec:
> If one or more of the communications can proceed, a single one that
> can proceed is chosen via a uniform pseudo-random selection.

Previously, this meant that some indeterminate amount of work could have
been completed after the context's cancel function had been called.

* Prevent recursion of discovery guess

Successful guesses from the wordlist or a pattern will have discovery
patterns generated based on them and successful discovery guesses will
not. Further processing should require human curation to avoid
automatically generating an unbounded amount of traffic.

Also fixes reading the wordlist from standard in by making it more like
reading from a file now that we have dynamic progress updates.

* Add option for arbitrary discovery patterns

* Test only patterns if provided a pattern file

* Update discover pattern option description
@Freeedim
Copy link

Freeedim commented Apr 4, 2025

Hi there!

I am running into an issue that might be related to this, so I haven't filed a new issue; please tell me if I should.

The issue happens with v3.6 when I use --wordlist-offset and -x like in the command:
gobuster dir -u $url -w 1000-words.txt -r -x pdf,txt --wordlist-offset $X

If $X == 0, the progress shows the number of attempts against 3000 (I assume it comes from the 1000 words in the wordlist, times 3 because each word is tried with two different extensions and with no extension).

I struggled to find how to only try the second half of the wordlist:

  • If I try $X = 1501, I have an error "offset is greater than the number of lines in the wordlist".
  • If I try $X = 501, gobuster correctly tries the expected 1500 tests, but displays progress in a weird way:
    • the progress is shown against 3000 total attempts (fine);
    • the initial value is 500 (where, against 3000, I would have expected 1501);
    • the final value, when gobuster stops and exits, is 2000 / 3000 (66.67%), which is a bit odd, although consistent with the 1500 tests.

In the end, the work is done, but in my use case (many VMs with different offsets on very long wordlists), it is very misleading and difficult to know when a given VM starts to overlap the tests already done by another VM that had a greater offset.

My suggestion is to either:

  • make shown progress only rely on the wordlist's number of words, and somehow indicate that each will result in N actual tests; or
  • make shown progress fully rely on the actual number of tests including, in case there is a --wordlist-offset and some -x file extensions, a starting value being the actual number of tests that were skipped instead of the number of words that were skipped.

Additionally, I would find it great to be able to ignore the N last words of a wordlist, or maybe replace --wordlist-offset by something like --wordlist-slice <range>, where <range> might be "100-200", "100-" or "-200".

@firefart
Copy link
Collaborator

firefart commented Apr 4, 2025

@Freeedim can you please try the dev branch? go install github.com/OJ/gobuster/v3@dev

There are a few updates to the progress handling in this version, curious if this problem still exists in the new code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants