You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
g++ -std=c++11 -pthread -O3 fastBPE/main.cc -IfastBPE -o fast
g++: error: fastBPE/main.cc: No such file or directory
g++: fatal error: no input files
compilation terminated.
workaround suggested under pytorch / fairseq error #1224 is also not working
Followed @zjplab instructions
1.Downloaded fastBPE repo
2.added "if msc_ver >= '1900': return ['vcruntime140']" in distutils\cygwinccompiler.py
3. created files mman.c,mman.h file with the code that you shared
4.replaced #include <sys/mman.h> with #include "mman.c" in fastBPE.hpp
Im getting this error 'thread' was not declared in this scope 28 | const size_t kThreads = max(1, min(10, int(thread::hardware_concurrency())))
Full stack trace: Tired to debug and search for this issue in other portals but was not able to get any solutions could you please assist
python setup.py build --compiler=mingw32 running build running build_py running build_ext building 'fastBPE' extension C:\MinGW\bin\gcc.exe -mdll -O -Wall -IfastBPE -IC:\Users\aa\AppData\Local\Programs\Python\Python37\include -IC:\Users\aa\AppData\Local\Programs\Python\Python37\include -c fastBPE/fastBPE.cpp -o build\temp.win-amd64-3.7\Release\fastbpe\fastbpe.o -std=c++11 -Ofast -pthread In file included from fastBPE/fastBPE.cpp:653: fastBPE/fastBPE.hpp:28:44: error: 'thread' was not declared in this scope 28 | const size_t kThreads = max(1, min(10, int(thread::hardware_concurrency()))); | ^~~~~~ fastBPE/fastBPE.hpp:20:1: note: 'std::thread' is defined in header ''; did you forget to '#include '? 19 | #include <unordered_set> +++ |+#include 20 | #include fastBPE/fastBPE.hpp: In function 'void fastBPE::readText(const char*, std::unordered_map<std::__cxx11::basic_string, unsigned int>&)': fastBPE/fastBPE.hpp:83:27: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t' {aka 'long long unsigned int'} [-Wformat=] 83 | fprintf(stderr, "Read %lu words (%lu unique) from text file.\n", total, | ~~^ ~~~~~ | | | | long unsigned int uint64_t {aka long long unsigned int} | %I64u fastBPE/fastBPE.hpp:83:38: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'std::unordered_map<std::__cxx11::basic_string, unsigned int>::size_type' {aka 'unsigned int'} [-Wformat=] 83 | fprintf(stderr, "Read %lu words (%lu unique) from text file.\n", total, | ~~^ | | | long unsigned int | %u 84 | word_count.size()); | ~~~~~~~~~~~~~~~~~ | | | std::unordered_map<std::__cxx11::basic_string, unsigned int>::size_type {aka unsigned int} fastBPE/fastBPE.hpp: In function 'void fastBPE::outputText(const char*, const char*, std::unordered_map<std::__cxx11::basic_string, std::__cxx11::basic_string >&)': fastBPE/fastBPE.hpp:136:7: error: 'ftruncate' was not declared in this scope; did you mean 'strncat'? 136 | if (ftruncate(fdOut, out_size) < 0) { | ^~~~~~~~~ | strncat fastBPE/fastBPE.hpp:137:65: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'size_t' {aka 'unsigned int'} [-Wformat=] 137 | fprintf(stderr, "Couldn't truncate output file %s to size %lu\n", fpo, | ~~^ | | | long unsigned int | %u 138 | out_size); | ~~~~~~~~ | | | size_t {aka unsigned int} fastBPE/fastBPE.hpp:149:31: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'long long unsigned int' [-Wformat=] 149 | fprintf(stderr, "Modified %lu words from text file.\n", p.second); | ~~^ ~~~~~~~~ | | | | long unsigned int long long unsigned int | %I64u fastBPE/fastBPE.hpp: In function 'void fastBPE::readVocab(const char*, std::unordered_map<std::__cxx11::basic_string, unsigned int>&)': fastBPE/fastBPE.hpp:438:27: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t' {aka 'long long unsigned int'} [-Wformat=] 438 | fprintf(stderr, "Read %lu words (%lu unique) from vocabulary file.\n", total, | ~~^ ~~~~~ | | | | long unsigned int uint64_t {aka long long unsigned int} | %I64u fastBPE/fastBPE.hpp:438:38: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'std::unordered_map<std::__cxx11::basic_string, unsigned int>::size_type' {aka 'unsigned int'} [-Wformat=] 438 | fprintf(stderr, "Read %lu words (%lu unique) from vocabulary file.\n", total, | ~~^ | | | long unsigned int | %u 439 | vocab.size()); | ~~~~~~~~~~~~ | | | std::unordered_map<std::__cxx11::basic_string, unsigned int>::size_type {aka unsigned int} fastBPE/fastBPE.hpp: In function 'void fastBPE::readCodes(const char*, std::unordered_map<std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string >, unsigned int, fastBPE::pair_hash>&, std::unordered_map<std::__cxx11::basic_string, std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string > >&)': fastBPE/fastBPE.hpp:462:27: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'std::unordered_map<std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string >, unsigned int, fastBPE::pair_hash>::size_type' {aka 'unsigned int'} [-Wformat=] 462 | fprintf(stderr, "Read %lu codes from the codes file.\n", codes.size()); | ~~^ ~~~~~~~~~~~~ | | | | long unsigned int std::unordered_map<std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string >, unsigned int, fastBPE::pair_hash>::size_type {aka unsigned int} | %u fastBPE/fastBPE.hpp: In function 'void fastBPE::applybpe(const char*, const char*, const char*, const char*)': fastBPE/fastBPE.hpp:607:37: error: size of array 'bpe' is not an integral constant-expression 607 | unordered_map<string, string> bpe[kThreads]; | ^~~~~~~~ fastBPE/fastBPE.hpp:608:10: error: 'thread' was not declared in this scope 608 | vector threads; | ^~~~~~ fastBPE/fastBPE.hpp:608:10: note: 'std::thread' is defined in header ''; did you forget to '#include '? fastBPE/fastBPE.hpp:608:16: error: template argument 1 is invalid 608 | vector threads; | ^ fastBPE/fastBPE.hpp:608:16: error: template argument 2 is invalid fastBPE/fastBPE.hpp:610:13: error: request for member 'emplace_back' in 'threads', which is of non-class type 'int' 610 | threads.emplace_back( | ^~~~~~~~~~~~ fastBPE/fastBPE.hpp:623:14: error: invalid types 'int[size_t {aka unsigned int}]' for array subscript 623 | threads[i].join(); | ^ error: command 'C:\MinGW\bin\gcc.exe' failed with exit status 1
The text was updated successfully, but these errors were encountered:
g++ -std=c++11 -pthread -O3 fastBPE/main.cc -IfastBPE -o fast
g++: error: fastBPE/main.cc: No such file or directory
g++: fatal error: no input files
compilation terminated.
workaround suggested under pytorch / fairseq error #1224 is also not working
Followed @zjplab instructions
1.Downloaded fastBPE repo
2.added "if msc_ver >= '1900': return ['vcruntime140']" in distutils\cygwinccompiler.py
3. created files mman.c,mman.h file with the code that you shared
4.replaced #include <sys/mman.h> with #include "mman.c" in fastBPE.hpp
Im getting this error 'thread' was not declared in this scope 28 | const size_t kThreads = max(1, min(10, int(thread::hardware_concurrency())))
Full stack trace: Tired to debug and search for this issue in other portals but was not able to get any solutions could you please assist
python setup.py build --compiler=mingw32 running build running build_py running build_ext building 'fastBPE' extension C:\MinGW\bin\gcc.exe -mdll -O -Wall -IfastBPE -IC:\Users\aa\AppData\Local\Programs\Python\Python37\include -IC:\Users\aa\AppData\Local\Programs\Python\Python37\include -c fastBPE/fastBPE.cpp -o build\temp.win-amd64-3.7\Release\fastbpe\fastbpe.o -std=c++11 -Ofast -pthread In file included from fastBPE/fastBPE.cpp:653: fastBPE/fastBPE.hpp:28:44: error: 'thread' was not declared in this scope 28 | const size_t kThreads = max(1, min(10, int(thread::hardware_concurrency()))); | ^~~~~~ fastBPE/fastBPE.hpp:20:1: note: 'std::thread' is defined in header ''; did you forget to '#include '? 19 | #include <unordered_set> +++ |+#include 20 | #include fastBPE/fastBPE.hpp: In function 'void fastBPE::readText(const char*, std::unordered_map<std::__cxx11::basic_string, unsigned int>&)': fastBPE/fastBPE.hpp:83:27: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t' {aka 'long long unsigned int'} [-Wformat=] 83 | fprintf(stderr, "Read %lu words (%lu unique) from text file.\n", total, | ~~^ ~~~~~ | | | | long unsigned int uint64_t {aka long long unsigned int} | %I64u fastBPE/fastBPE.hpp:83:38: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'std::unordered_map<std::__cxx11::basic_string, unsigned int>::size_type' {aka 'unsigned int'} [-Wformat=] 83 | fprintf(stderr, "Read %lu words (%lu unique) from text file.\n", total, | ~~^ | | | long unsigned int | %u 84 | word_count.size()); | ~~~~~~~~~~~~~~~~~ | | | std::unordered_map<std::__cxx11::basic_string, unsigned int>::size_type {aka unsigned int} fastBPE/fastBPE.hpp: In function 'void fastBPE::outputText(const char*, const char*, std::unordered_map<std::__cxx11::basic_string, std::__cxx11::basic_string >&)': fastBPE/fastBPE.hpp:136:7: error: 'ftruncate' was not declared in this scope; did you mean 'strncat'? 136 | if (ftruncate(fdOut, out_size) < 0) { | ^~~~~~~~~ | strncat fastBPE/fastBPE.hpp:137:65: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'size_t' {aka 'unsigned int'} [-Wformat=] 137 | fprintf(stderr, "Couldn't truncate output file %s to size %lu\n", fpo, | ~~^ | | | long unsigned int | %u 138 | out_size); | ~~~~~~~~ | | | size_t {aka unsigned int} fastBPE/fastBPE.hpp:149:31: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'long long unsigned int' [-Wformat=] 149 | fprintf(stderr, "Modified %lu words from text file.\n", p.second); | ~~^ ~~~~~~~~ | | | | long unsigned int long long unsigned int | %I64u fastBPE/fastBPE.hpp: In function 'void fastBPE::readVocab(const char*, std::unordered_map<std::__cxx11::basic_string, unsigned int>&)': fastBPE/fastBPE.hpp:438:27: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t' {aka 'long long unsigned int'} [-Wformat=] 438 | fprintf(stderr, "Read %lu words (%lu unique) from vocabulary file.\n", total, | ~~^ ~~~~~ | | | | long unsigned int uint64_t {aka long long unsigned int} | %I64u fastBPE/fastBPE.hpp:438:38: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'std::unordered_map<std::__cxx11::basic_string, unsigned int>::size_type' {aka 'unsigned int'} [-Wformat=] 438 | fprintf(stderr, "Read %lu words (%lu unique) from vocabulary file.\n", total, | ~~^ | | | long unsigned int | %u 439 | vocab.size()); | ~~~~~~~~~~~~ | | | std::unordered_map<std::__cxx11::basic_string, unsigned int>::size_type {aka unsigned int} fastBPE/fastBPE.hpp: In function 'void fastBPE::readCodes(const char*, std::unordered_map<std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string >, unsigned int, fastBPE::pair_hash>&, std::unordered_map<std::__cxx11::basic_string, std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string > >&)': fastBPE/fastBPE.hpp:462:27: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'std::unordered_map<std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string >, unsigned int, fastBPE::pair_hash>::size_type' {aka 'unsigned int'} [-Wformat=] 462 | fprintf(stderr, "Read %lu codes from the codes file.\n", codes.size()); | ~~^ ~~~~~~~~~~~~ | | | | long unsigned int std::unordered_map<std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string >, unsigned int, fastBPE::pair_hash>::size_type {aka unsigned int} | %u fastBPE/fastBPE.hpp: In function 'void fastBPE::applybpe(const char*, const char*, const char*, const char*)': fastBPE/fastBPE.hpp:607:37: error: size of array 'bpe' is not an integral constant-expression 607 | unordered_map<string, string> bpe[kThreads]; | ^~~~~~~~ fastBPE/fastBPE.hpp:608:10: error: 'thread' was not declared in this scope 608 | vector threads; | ^~~~~~ fastBPE/fastBPE.hpp:608:10: note: 'std::thread' is defined in header ''; did you forget to '#include '? fastBPE/fastBPE.hpp:608:16: error: template argument 1 is invalid 608 | vector threads; | ^ fastBPE/fastBPE.hpp:608:16: error: template argument 2 is invalid fastBPE/fastBPE.hpp:610:13: error: request for member 'emplace_back' in 'threads', which is of non-class type 'int' 610 | threads.emplace_back( | ^~~~~~~~~~~~ fastBPE/fastBPE.hpp:623:14: error: invalid types 'int[size_t {aka unsigned int}]' for array subscript 623 | threads[i].join(); | ^ error: command 'C:\MinGW\bin\gcc.exe' failed with exit status 1
The text was updated successfully, but these errors were encountered: