Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

brew update/brew upgrade failing with Error: Not a directory @ dir_s_rmdir error #1811

Closed
1 of 6 tasks
traversaro opened this issue Oct 14, 2020 · 27 comments
Closed
1 of 6 tasks
Assignees
Labels
Area: Apple Area: Packages investigate Collect additional information, like space on disk, other tool incompatibilities etc.

Comments

@traversaro
Copy link

Description
On some scheduled jobs on macOS, since few days the brew update and brew upgrade are failing.

Area for Triage:

Question, Bug, or Feature?:

Bug

Virtual environments affected

  • macOS 10.15
  • Ubuntu 16.04 LTS
  • Ubuntu 18.04 LTS
  • Ubuntu 20.04 LTS
  • Windows Server 2016 R2
  • Windows Server 2019

Expected behavior
The commands:

  • brew update
  • brew upgrade

Should work fine.

Actual behavior

  • brew update
  • brew upgrade
    commands fail with error:
==> Summary
🍺  /usr/local/Cellar/node@12/12.19.0_1: 4,304 files, 59.5MB
Removing: /usr/local/Cellar/node@12/12.18.4... (4,296 files, 58.9MB)
Removing: /Users/runner/Library/Caches/Homebrew/[email protected]... (15.8MB)
Error: Not a directory @ dir_s_rmdir - /usr/local/Cellar/openssl
==> Upgrading [email protected] 1.1.1h -> 1.1.1h 
Removing: /usr/local/Cellar/openssl/1.0.2t... (1,787 files, 12.4MB)
Error: Process completed with exit code 1.

Repro steps
A description with steps to reproduce the issue. If your have a public example or repo to share,
please provide the link.

  1. Run brew update
  2. Run brew upgrade
@traversaro
Copy link
Author

I do not have a mac so I can't try to reproduce this, but I checked online and it seems that it is affecting other CI systems: dotnet/runtime#43299 .

@traversaro
Copy link
Author

@LeonidLapshin LeonidLapshin added Area: Apple Area: Packages investigate Collect additional information, like space on disk, other tool incompatibilities etc. and removed needs triage labels Oct 14, 2020
@LeonidLapshin LeonidLapshin self-assigned this Oct 14, 2020
@LeonidLapshin
Copy link
Contributor

Hey, @traversaro !
It seems that openssl (Cellar/openssl/1.0.2t) that is the cause of the error will be removed from macOS images soon, the link to corresponding PR
for now you can workaround this issue with

brew uninstall [email protected]

command before

brew update
brew upgrade

I have tried to install your dependencies from https://github.com/robotology/robotology-superbuild/blob/master/.github/workflows/ci.yml and

          brew uninstall [email protected]
          brew cask install xquartz
          brew update
          brew upgrade
          brew install ace boost cmake eigen gsl ipopt jpeg libedit opencv pkg-config qt5 sqlite swig tinyxml
          brew install libmatio
          brew tap osrf/simulation
          brew install osrf/simulation/gazebo11
          brew install ninja

works correctly.
Could you please check the provided workaround?
Thank you!

@traversaro
Copy link
Author

Thanks!

traversaro added a commit to robotology/robotology-superbuild that referenced this issue Oct 14, 2020
Workaround mentioned in actions/runner-images#1811 .
@traversaro
Copy link
Author

Thanks a lot! I tried the workaround in robotology/robotology-superbuild#493 , but doing that in my CI results in this failure:

fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Fetching /usr/local/Homebrew/Library/Taps/local/homebrew-openssl failed!
Fetching /usr/local/Homebrew/Library/Taps/local/homebrew-python2 failed!
fatal: invalid upstream 'origin/master'
fatal: invalid upstream 'origin/master'

@LeonidLapshin
Copy link
Contributor

@traversaro , seems that happens because of homebrew version, could you please try to run:

brew uninstall [email protected]
brew uninstall [email protected]
brew untap local/openssl
brew untap local/python2
brew cask install xquartz
brew update
brew upgrade
brew install ace boost cmake eigen gsl ipopt jpeg libedit opencv pkg-config qt5 sqlite swig tinyxml
...

it should help.
Thank you!

@traversaro
Copy link
Author

Thanks a lot @LeonidLapshin , your workaround is working fine!

lizan added a commit to lizan/envoy that referenced this issue Oct 14, 2020
htuch pushed a commit to envoyproxy/envoy that referenced this issue Oct 14, 2020
mpuncel added a commit to mpuncel/envoy that referenced this issue Oct 15, 2020
* master:
  ci: use multiple stage (envoyproxy#13557)
  tls: update BoringSSL to 2192bbc8 (4240). (envoyproxy#13567)
  fix macos v8 build (envoyproxy#13572)
  Fixed Health Check Fuzz corpus syntax (envoyproxy#13576)
  ci: Remove shellcheck diff (envoyproxy#13560)
  ci: Increate brew retry interval (envoyproxy#13565)
  dependencies: fix some of the fallout from Wasm merge. (envoyproxy#13569)
  hds: add support for delta updates in specifier (envoyproxy#13067)
  ci: workaround for actions/runner-images#1811 (envoyproxy#13577)
  ratelimit: be able to disable x-envoy-ratelimited response header sent (envoyproxy#13270)
  Update opencensus library (envoyproxy#13549)
  ci: use azp for api and go-control-plane sync (envoyproxy#13550)
  docs: Remove/make generic lyft references in docs (envoyproxy#13559)
  check_format: adding 2 more release note checks (envoyproxy#13444)
  [Wasm] Add cluster metadata fallback and upstream host metadata (envoyproxy#13477)
  [fuzz] Added validation for secrets (envoyproxy#13543)
  Add Platform Specific Feature guidance to PR template (envoyproxy#13547)

Signed-off-by: Michael Puncel <[email protected]>
Nelson-numerical-software added a commit to nelson-lang/nelson that referenced this issue Oct 20, 2020
rchen152 added a commit to google/pytype that referenced this issue Oct 20, 2020
helkhalfi added a commit to helkhalfi/dotfiles that referenced this issue Dec 7, 2020
alloy added a commit to microsoft/react-native-macos that referenced this issue Dec 8, 2020
alloy added a commit to microsoft/react-native-macos that referenced this issue Dec 8, 2020
alloy added a commit to NickGerleman/react-native that referenced this issue Dec 8, 2020
alloy added a commit to microsoft/react-native-macos that referenced this issue Dec 8, 2020
alloy added a commit to microsoft/react-native-macos that referenced this issue Dec 8, 2020
…eyboard… (#668)

* Avoid eating clicks/taps into ScrollView when using physical keyboard (facebook#30374)

Summary:
This is an extension of facebook#29798 which was reverted due to cases where the soft keyboard could not be dismissed.

## Changelog

<!-- Help reviewers and the release process by writing your own changelog entry. For an example, see:
https://github.com/facebook/react-native/wiki/Changelog
-->

[General] [Fixed] - Avoid eating clicks/taps into ScrollView when using physical keyboard

Pull Request resolved: facebook#30374

Test Plan: Validated with iOS simulator that taps on default ScrollView will dismiss soft keyboard and be eaten if open, but taps are not eaten when emulating a connected physical keyboard.

Reviewed By: kacieb

Differential Revision: D24935077

Pulled By: lyahdav

fbshipit-source-id: 19d9cf64547e40a35f9363896e3abbdccb95b546

* [ado] Workaround homebrew openssl issue

actions/runner-images#1811 (comment)

* [RNTester] Update pods

Co-authored-by: Eloy Durán <[email protected]>
pjcollins pushed a commit to dotnet/android that referenced this issue Dec 8, 2020
Context: actions/runner-images#1811

Recently some CI builds are ending early, because Homebrew is failing
like this:

    Installing automake
    ...
    stderr | Error: Not a directory @ dir_s_rmdir - /usr/local/Cellar/openssl
     | Removing: /usr/local/Cellar/openssl/1.0.2t... (1,787 files, 12MB)

    Error: Installation of automake failed

Apply the suggested workaround to fix this for now while we wait for
updated macOS images.
HeyImChris added a commit to microsoft/react-native-macos that referenced this issue Dec 12, 2020
* Update RCTCxxBridge.mm

* macOS keyboard support

* update comment

* add key value checks so we don't send all events

* macOS keyboard support

* update comment

* add key value checks so we don't send all events

* [ado] Workaround homebrew openssl issue (#669)

actions/runner-images#1811 (comment)

* Fix detox yarn error with Xcode 12 (#670)

* Update RCTCxxBridge.mm

* fix detox errors with xcode 12

* only use valid keys, bubble events to super

* macOS keyboard support

* update comment

* add key value checks so we don't send all events

* only use valid keys, bubble events to super

* macOS keyboard support

* add key value checks so we don't send all events

* resolve bad merge

* update valid key bug, api typo

* spacing fix

* fix flow errors

* fix snapshot tests for new APIs

* yarn lint --fix

* fix flipper

Co-authored-by: Eloy Durán <[email protected]>
harrieshin pushed a commit to microsoft/react-native-macos that referenced this issue Dec 16, 2020
* Update RCTCxxBridge.mm

* macOS keyboard support

* update comment

* add key value checks so we don't send all events

* macOS keyboard support

* update comment

* add key value checks so we don't send all events

* [ado] Workaround homebrew openssl issue (#669)

actions/runner-images#1811 (comment)

* Fix detox yarn error with Xcode 12 (#670)

* Update RCTCxxBridge.mm

* fix detox errors with xcode 12

* only use valid keys, bubble events to super

* macOS keyboard support

* update comment

* add key value checks so we don't send all events

* only use valid keys, bubble events to super

* macOS keyboard support

* add key value checks so we don't send all events

* resolve bad merge

* update valid key bug, api typo

* spacing fix

* fix flow errors

* fix snapshot tests for new APIs

* yarn lint --fix

* fix flipper

Co-authored-by: Eloy Durán <[email protected]>
stloeffler added a commit to stloeffler/texworks-testing that referenced this issue Dec 19, 2020
@636f7374
Copy link

Solution

  • The above methods are now invalid (not working).
  • After many tests, this can solve this problem.
  • Add this before brew update.

Action

    - name: TemporaryFix - Brew Package Management (Github/Action#1811)
      run: |
        brew uninstall --force --ignore-dependencies openssl
        brew cleanup -s openssl
        brew cleanup --prune-prefix
        brew install openssl
        brew uninstall --force --ignore-dependencies python
        rm $(which 2to3)
        brew cleanup -s python
        brew cleanup --prune-prefix
        brew install python

elopez added a commit to elopez/echidna that referenced this issue Jan 26, 2021
This has been fixed upstream by GitHub.
elopez added a commit to elopez/echidna that referenced this issue Jan 28, 2021
This has been fixed upstream by GitHub.
ggrieco-tob pushed a commit to crytic/echidna that referenced this issue Jan 28, 2021
* GitHub Actions: macOS releases: Drop extra libgmp renaming

* GitHub Actions: drop workaround for actions/runner-images#1811

This has been fixed upstream by GitHub.

* GitHub Actions: re-enable MacOS CI
ggrieco-tob added a commit to crytic/echidna that referenced this issue Feb 3, 2021
* semver integration for improve testing with different solc versions

* fixes

* more fixes

* GitHub Actions: macOS releases: Drop extra libgmp renaming

* GitHub Actions: drop workaround for actions/runner-images#1811

This has been fixed upstream by GitHub.

* GitHub Actions: upgrade cache and setup-python actions to v2

* GitHub Actions: prettify Linux artifact

Currently the tar.gz file has several folders inside from where
the binary used to be in the runner, clean those up using -C.

* Make tests static

This helps in the case of CI, and allows to run the tests on a separate
container without having to set up any library or Haskell dependencies.

* GitHub Actions: implement parallel testing

This allows for parallel execution of the echidna tests, which
helps improve testing speed, especially once newer solc versions
are included in the pipeline.

* GitHub Actions: install the exact solc needed instead of all versions

* GitHub Actions: re-enable MacOS CI

Co-authored-by: ggrieco-tob <[email protected]>
Co-authored-by: Gustavo Grieco <[email protected]>
@Patrick-Chong
Copy link

Patrick-Chong commented Jul 22, 2021

Hi, I am running into the same error when running 'brew update':

(venv) (base) patrickchong@Patricks-MacBook-Pro microblog % brew update
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Fetching /opt/homebrew/Library/Taps/company/homebrew-team failed!
fatal: 'origin/master' is not a commit and a branch 'master' cannot be created from it
fatal: invalid upstream 'origin/master'

I am using macOS Big Sur.

Any ideas how to fix this? I tried some of the above workarounds but still run into this error!

@rchawathearkin
Copy link

Same here. Even after following instructions from #1811 (comment)
Facing issue as follows

~ $brew update
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Fetching /usr/local/Homebrew/Library/Taps/rchawathe/homebrew-local-tap failed!
fatal: 'origin/master' is not a commit and a branch 'master' cannot be created from it
fatal: invalid upstream 'origin/master'

rexengineering pushed a commit to rexengineering/istio-envoy that referenced this issue Oct 15, 2021
@DhairyeKataria
Copy link

Same here. Even after following instructions from #1811 (comment) Facing issue as follows

~ $brew update
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Fetching /usr/local/Homebrew/Library/Taps/rchawathe/homebrew-local-tap failed!
fatal: 'origin/master' is not a commit and a branch 'master' cannot be created from it
fatal: invalid upstream 'origin/master'

Please Help If you've found the solution!!

Mine gives this output:

brew update
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core failed!

unlucku added a commit to unlucku/conscrypt that referenced this issue Mar 16, 2022
I believe the actions/runner-images#1811 has been resolved from Githubs end.
baibailiha added a commit to baibailiha/terraform-aws-eks that referenced this issue Sep 13, 2022
robertszafa pushed a commit to robertszafa/llvm-sycl-passes that referenced this issue Feb 17, 2023
This patch adds basic Travis support - it only installs LLVM 8 (which is
the version that will initially be used for this repo).

Fix Travis and add status in README.md

Add passes for counting direct function calls

Two following passes are added:
  * StaticCallCounter (counts static function calls)
  * DynamicCallCounter (counts dynamic function calls)

Also:
* documentation
* updated build system
* updated CI

Sort alphabetically output from StaticCallCounter

This makes the output deterministic and will make testing possible.

Add LIT test for StaticCallCounter

This patch contains one LIT test, as well as LIT configuration scripts
and documentation updates

Add syntax highlighting in README.md

Update the comments in the existing passes

Updated, added and refined comments for the exisiting passes. Also minor
refactoring (e.g. avoid implicitely casting pointers to bool).

Refactor the test

* Moved the source file to a dedicated location (so that it can be used as
both an example and an input for tests)
* Added comments
* Simplified the lit test (the source code was not required there)

[Darwin] Add support for OS X

* Fixes build on Darwin (updated compiler flags)
* Adds OS X in Travis
* Adds -fvisibility-inlines-hidden compiler flag to silence warnings on
Darwin and for consistency LLVM build system

Add the Mixed Boolean Arithmetic pass

There's 5 new LIT tests that verify that this pass is working fine. Some
clean-up, code re-ordering to be expected.

[NFC] Add instructions for building LLVM from sources

[NFC] Simplify README.md

[NFC] Update status

Rename the libraries

* simplifies the way libraries are called
* stop building static libs (currently not needed)
* [NFC] removes a duplicated test

[NFC] Update README.md

This patch also renames the `lt` pass to `static-cc` to better reflect
what it does. I'm qualifying this as NFC.

Refactor and add new tests

Renaming (e.g. *.c -> *.ll), more tests for CallCounter, comments.

[NFC] Simplify directory structure

Add the RIV pass

Add the DuplicateBB pass

Minor refactoring

* Remove the custom RandomNumberGenerator wrapper. The one in LLVM is
sufficient.
* Rename Utils.{cpp|h} --> Ratio.{cpp|h}. This is both cleaner and
better reflects what Ratio.{cpp|h} is for.

Add new Pass Manager support for MBAAdd

In this patch I also renamed MBA as MBAAdd as I'll be adding more passes
using mixed boolean algebra.

Add MBASub

Also tests, input file, updated README.md, renamed MBA to MBAAdd.

Refactor MBAAdd (use more complicated formula)

This patch updates the MBAAdd pass by replacing the original formula:
  (1)  a + b == (a ^ b) + 2 * (a & b)
with a bit more involved one:
  (2)  a + b ==(((a ^ b) + 2 * (a & b)) * 39 + 23) * 151 + 111
Since (2) is only valid for 8 bit integers, MBAAdd will now only
transform add instructions when both inputs are 8 bit wide.

Tests and README.md are updated accordingly. Since MBAAdd will now only
work for additions of 8-bit integers, e.g.
  %3 = add i8 %1, %2
I had to play a bit with examples so that there are not sign-extensions
to 32 bit integers (e.g. compiling with -O1).

All this hopefully introduced some interesting concepts.

Fix the way the return conditionin MBASub::run is calculated

Thanks MattPD for pointing this out!

[NFC] Refactor lt-cc-main.cpp

Simplified the code, added comments, made sure that formatting adheres
to LLVM coding style.

Refactor StaticCallCounters and DynamicCallCounters

* split lt-cc-main.cpp into StaticMain.cpp and DynamicMain.cpp
* rename files to adhere to LLVM's coding style
* rename variables to adhere to LLVM's coding style
* remove unnecessary includes
* simplified the code (e.g. removed a few command line options)
* more comments
* updated README.md and tests accordingly
* remove the tools/lt-cc dir

[NFC] Apply clang-format

Add HelloWorld pass

Adding HelloWorld - a reference pass with a self-contained
CMakeLists.txt file. It's a rather minimal example that should be a good
starting point. Updated README.md accordingly.

[NFC] Update README.md

[NFC] Fix typo

[CMake] Simplify the scripts

This is mostly non-functional-changes, apart from removing a shared
library target: DynamicCallCounter. It isn't meant to be used as a
shared object, so that target was redundant.

[NFC] Update Credits

Update the HelloWorld example

Add a test that verifies registration with the new PM.

Update comments with regard to the known seg-fault on Travis. I realised
that I can reproduce it locally when building against packages from
HomeBrew. I've update the comments accordingly. I'm still not aware of
any working solutions.

Update HelloWorld and the CMake flag that guards it from crashing

1. Renamed LT_TRAVIS_DISABLE to LT_LEGACY_SKIP_PIPELINE_REGISTRATION
It's no obvious that it's registration within one of the pre-existing
pipelines (using the legacy PM mechanism) causes 'opt' to segfault when
building against LLVM from HomeBrew. The new name of the CMake variable
reflects that.

2. [NFC] Updated comments in HelloWorld

[NFC] Update documentaiton for the MBAAdd pass

[CMake] Fix the way CMAKE_CXX_FLAGS are set

CMAKE_CXX_FLAGS were set using CMAKE_CXX_FLAGS_RELEASE, which means that
regardless of CMAKE_BUILD_TYPE, Release build flags were used.

[NFC] Update README.md

[NFC] Refactor MBAAdd and MBASub

* Remove redundant variables (the instruction iterator from the 'for'
  loop wasn't used directly, now it is)
* Updated comments, moved code around (for consistency)
* Removed comments which are already present in HelloWorld (to minimise
  duplication)

Switch StaticCallCounter to the new PM

* the new PM is now the default for StaticCallCounter
* switched to the new PM in StaticMain.cpp too
* refactored StaticCallCounter to make it consistent with other passes
  already using the new PM

Note that the unit tests still require to run StaticCallCounter via the
legacy PM, as the passes in the new PM don't implement the print method:
  http://llvm.org/docs/WritingAnLLVMPass.html#the-print-method

Update LIT config for clang

Until now clang wasn't listed with other test tools in lit.cfg.py, which
meant that it's path wasn't prepended with LT_LIT_TOOLS_DIR. In other
words, the system clang was always used. This patch makes sure that
clang from LT_LIT_TOOLS_DIR is used instead. Provided that
LT_LIT_TOOLS_DIR is set correctly, this guarantees that clang-8 is
always used.

On Darwin clang installed via HomeBrew (or build from sources) doesn't
know where to look for standard headers (e.g. 'stdlib.h'). The
workaround implemented here  uses '-isysroot'.

[NFC] Refactor DynamicCallCounter and dynamic

These changes are meant to simplify and clarify the code:
* variable renaming
* updated comments
* code simplification.

Update to LLVM 9

* Updated source code (rather minor changes)
* Updated .travis.yml (also switched from Ubuntu Xenial to Ubuntu
  Bionic)
* Update README.md

I believe that there are missing files in the LLVM 9 package for Ubuntu
(or LLVMExport.cmake script is broken). I've managed to work around in
.travis.yml, but need to investigate further. I'll need to set-up an
Ubuntu VM for this first.

[Travis] Enable LIT testing on Linux

LLVM-9 packages for Ubuntu ship with FileCheck, which means that I can
now run LIT tests there too (more specifically, on Ubuntu nodes on
Travis).

Some tests required updating - the order of instructions on Ubuntu was
different.

[NFC] Simplify test/hello.ll

[NFC] Update the NewInst generation in MBAAdd

This is just to improve the readability

[Darwin/NFC] Update the way the path for -isysroot is discovered

This path is now discovered with xcrun as opposed being hard-coded.

[NFC/CMake] Minor simplification

* HelloWorld doesn't require link_directories - removed
* HelloWorld doesn't require add_definitions - removed
* LLVM 9 switched to C++14, so removed LLVM_ENABLE_CXX1Y

[NFC/CMake] Remove redundant stuff

[NFC] Rename variables (to upper case) and remove redundant func args

[NFC/LIT] Remove redundant code

[NFC] Fix format specifier in printf

[NFC] Simplify MBA_add.ll and MBA_sub.ll

Instead of using external input C files, I generated the input IR
function and copied it verbatim (module the meta data) into the test
files. This allowed me to remove the RUN line that would execute
'clang'. The tests are now better focused on what should be tested, i.e.
IR --> IR transformation.

Add a test for MBAAdd

This patch adds a test for MBAAdd that verifies that 32-bit additions
are not transformed. The original test was renamed to reflect that it
only tests 8-bit additions.

[NFC] Refine CMake, remove redundant suffic in LIT config

[NFC] Update README.md (re switch to LLVM 9)

With the switch to LLVM 9,  LIT tests can now be run in the Ubuntu
Travis instance too. This patch updates README accordingly.

Also, the CMake variable LT_LIT_TOOLS_DIR isn't really needed, so I
removed to simplify the instructions.

Simplify tests for call counting passes

Typo in Filename in README

[NFC] Add hints about debugging

Fix some incorrect directories in README.md

[Travis] Remove '-va' from command line when running LIT

Also make sure that apt-get update is always run before installing LLVM
(apt-get can get flaky otherwise).

Remove DynamicCallCounter

I would like to simplify the examples and this one in particular
requires breaking down.

Fix bug in README.md

The original command was loading the plugin using the new PM style (via
`-load-pass-plugin`), yet was trying to use it through the legacy PM style
(via `-hello-world`).

[NFC] Update comments

[NFC] Fix wrong pass names in README.md

Thank you @llVim!

[NFC] Add references

[NFC] Documentation

[Docs] Fix typo in installation instructions for Ubuntu

[CI] Add *.yaml files for GitHub Actions

Add InjectFuncCall pass

This pass injects calls to printf (from stdio.h) into the input module.

[CI] Clean-up the Travis config file

* Remove workarounds that are no longer needed
* Remove code that's no longer used/needed (e.g. $RUN_TESTS)
* Update comments

Update attributes for printf in InjectFuncCall

Based on the suggestion from MattPD, I've updated the attributes that
are set for `printf` in InjectFuncCall.cpp. It now matches the
implementation of `inferLibFuncAttributes` in BuildLibCalls.cpp.

It would be tempting to call `inferLibFuncAttributes` instead, but that
hook requires an instance of TargetLibraryInfo. Although currently it
_can_ be obtained through ModuleAnalysisManager, this is about to change
in LLVM 10:

https://reviews.llvm.org/D66428

I've also switched from `PointerType::get` to `PointerType::getUnqual` -
this allows to pass fewer arguments. Kudos to MattPD for the suggestion!

[CMake] Refactor the scripts

This patch adds a bunch of non-functional changes:
  * remove code that's not needed
  * re-arrange a bit
  * add comments

[CI] Make sure README.md displays the status of the master branch

Refine the badge for GitHub actions so that it always displays the
status of the master branch (as opposed to the branch currently being
developed/pushed to).

Add DynamicCallCounter pass

Counts dynamic (i.e. runtime) calls to functions defined in the input
module. To this end, DynamicCallCounter instruments the input module
with instructions that count runtime function calls. It also adds code
that prints a small report/summary after `main`.

Update and add tests for DynamicCallCounter

* New test that checks the output IR
* More comments
* Renamed the files

Update tests for StaticCallCounter

* Rename for consistency with DynamicCallCounter
* Use LLVM IR (instead of *.c) as input

Make sure DynamicCallCounter doesn't instrument empty files

DynamicCallCounter should only instrument function definitions (as
opposed to function declarations). Modules with no function definitions
should remain unmodified.

Add test/Inputs directory with re-usable input files for tests

To avoid code-duplication, add test/Inputs with input IR files re-used
by multiple tests.

Remove hard-coded paths from tests

Instead of using hard-coded paths for plugins (e.g. "..\lib"), use the
CMake variable, CMAKE_LIBRARY_OUTPUT_DIRECTORY.

Unify the examples for HelloWorld and InjectFuncCall

Although the two passes are completely differently, it's very
interesting to compare the output when applied to the same file.

This patch adds a few tweaks which makes the comparison straightforward.

[NFC] Update README.md

Add `About` section

[NFC] Update README.md

* Some re-writing
* Consistency
* Uniform formatting

Rework pipeline registration example

The HelloWorld pass includes registration with the optimisation
pipelines (on top of regular registration implemented for all other
examples). That registration has been guarded with a CMake flag to
optionally disable it. This is required on MacOS as for `opt` installed
with HomeBrew, registering out-of-tree passes with optimisation
pipelines leads to a seg-fault. Currently, in order for that coded to be
disabled one has to specify:
  LT_LEGACY_SKIP_PIPELINE_REGISTRATION=On
when calling CMake. In other words, the problematic
code is _enabled_ by default. It would make more sense to have it
_disabled_ by default so that it never causes an accidental seg-fault.

With this patch, the problematic registration is disabled by default.
LT_LEGACY_SKIP_PIPELINE_REGISTRATION is replaced with
HELLOWORLD_OPT_PIPELINE_REG, which is to be set to ON when requesting
registration with the optimisation pipelines.

This was suggested by Amirahmad Khordadi in issue banach-space#8 - thank you!

README.md was also updated to reflect the changes and with a few general
edits.

[CI/NFC] Update comments, remove unused CMake variable

Add links to LLVM Dev 2019 talks

[CI] Add new workflows for building LLVM 9 from sources

This patch adds GitHub Actions workflows that build LLVM 9 from sources
for both Ubuntu and Mac OS. In the current set-up these workflows are
triggered on every push. This is required in order to generate initial
data for the master branch. This is meant to be updated so that these
workflows are only run on certain times. Otherwise they are too
time-consuming and will eat-up all free GitHub Actions time.

This patch also includes some minor refactoring for the original
workflows.

[CI] Make sure LLVM is build only on certain dates

The GitHub Actions workflows that build LLVM sources take a lot of time
and can't be run every time code is pushed. This patch updates the
configuration so that these jobs run only on selected dates at specified
times.

[CI] Make sure LLVM is build only on certain dates - Darwin

Same as previous commit. I forgot to add
x86-darwin-llvm-from-sources.yml, so fixing it now.

[CI] Enable auto-registration with the existing pass pipelines

In order to increase test coverage, I'm enabling auto-registration with
the existing optimisation pipelines. Note that this is only implemented
for HelloWorld.

This patch only updates GitHub Actions. I might be making similar change
in travis.yml in the future.

Keep in mind that this auto-registration can only be enabled on Darwin
when building LLVM from sources (the reasons for this are documented in
the code).

[NFC] Remove redundant includes

Remove auto-registration from HelloWorld

Auto-registration with the existing pipelines (through
`RegisterStandardPasses`) complicates HelloWorld. I'd like to keep it
very simple instead. In this patch I'm removing auto-registration. I'll
create a dedicated pass for that and submit in a separate patch.

Additionally, I've made a few NFCs (code clean-up).

Add OpcodeCounter pass

It's a relatively simple pass, only slightly more complicated than
HelloWorld. However, I'd like to use it to showcase how to auto-register
passes. I think that it is worthwhile to have a dedicated, non-trivial
example for this. HelloWorld will remain as a basic reference example.

Fix links in README.md

[NFC] Move the definition of AnalysisKey StaticCallCounter::Key

This is just a consistency fix

Rewrite the RIV pass

The original implementation of RIV had a few issues:
  * it contained bugs (FP values weren't excluded as they should)
  * it didn't take into account global variables
  * it didn't have a new PM interface
  * the sample input file was too complex
  * the comments in the implementation were confusing
  * the documentation in README.md didn't highlight the key features
  * there was only 1 test

This patch addresses all of the above points. In particular, it
emphasises that RIV is the only pass in llvm-tutor that relies on
another pass from LLVM. In particular, it leverages LLVM's dominance
analysis.

Fix RIV pass

1. Use FAM.getResult<DominatorTreeAnalysis>(F) instead of
FAM.getCachedResult<DominatorTreeAnalysis>(F). The latter should only be
used when the results are already available. This only affect the new PM
implementation.

2. When adding a global variable to an RIV set, make sure to add the
global itself rather than just the underlying value.

3. All names should start with an upper case (global -> Global)

[NewPM] Prevent RIV from printing the results unconditionally

For the legacy PM one has to pass `-analyze` on the command line for the
results of an analysis pass to be printed. Unfortunately there's no
equivalent flag for the new PM. In order to test RIV with the new PM,
I've set it up so that it would print the results unconditionally.
That's not desirable though, so I'm reverting that behaviour.

Rewrite the DuplicateBB pass

This patch rewrites DuplicateBB:
  * label duplicated blocks so that it's clear which block have been
    injected by DuplicateBB, e.g. `lt-if-then-0` and `lt-else-0` for the
    0th BasicBlock that's duplicated
  * simplify the implementation (e.g. split into more functions, use
    more meaningful variable names, no command-line arguments)
  * make sure that global variables are not used for the injected
    `if-then-else` construct (and be explicit about it)
  * add new PM implementation
  * add more tests
  * add a simplified sample input
  * improved documentation

Add tests for DuplicateBB

These tests verify various corner cases. Also, they illustrate the
expected output in those cases.

[NewPM] Add auto-registration for OpcodeCounter

This patch updates the OpcodeCounter example to use auto-registration
when using the new PM implementation. AFAIK, with the new PM one can
either choose the auto-registration or the command-line registration.
Contrary to the legacy PM, it's not possible to use both for the same
pass.

Update DuplicateBB (including the description)

Make DuplicateBB name all the newly inserted basic blocks.

[Travis] Stop reporting results from Travis

My Travis instance has started acting up. I'm not sure whether that's
Travis or my Travis config. GitHub Actions is fine, so definitely not
llvm-tutor. I'm turning off notifications from Travis while I
investigate.

[Travis] Refactor the config

The multi-os Travis config has recently started failing on Darwin with
no apparent reason, i.e. there have been no changes in the build
scripts that could trigger this.

This is the (rather vague) error message from Travis:
"An error occurred while generating the build script."
  https://travis-ci.org/github/banach-space/llvm-tutor/jobs/665338881

This patch updates the Travis script so that:
  * instead of building on multiple os'es, multiple compilers are used
  (gcc and clang)
  * the os is set to Linux Bionic

Build in Debug _and_ Release

Until now all CI configs used the default build type. This patch will
make Travis build both Debug (the default) and Release.

This patch made me realise that I had a small bug in DuplicateBB that
manifested itself only when building in release, as then LLVM's
STATISTIC macro generates a variable that's always set to 0. DuplicateBB
is updated accordingly.

[NFC] Remove duplicated test

auto_registration.ll is identical to auto_registration_legacy_pm.ll

[NFC] Update comments in DuplicateBB

Add the MergeBB pass

[NFC] Add documentation for MergeBB

[NFC] Make sure that tests use %t

As per https://llvm.org/docs/CommandGuide/lit.html#substitutions, %t is
a common substitution used for temporary file name unique to the test.
Make sure that all tests use %t instead of some arbitrary names. This is
important as otherwise tests might be re-using files generated by other
tests.

[CMake] Remove unused variable definition

Upgrade to LLVM 10

The `LT_OPT_PIPELINE_REG` CMake flag and the corresponding workaround for
pass auto-registration has been removed. The corresponding issue in LLVM
has been fixed:
llvm/llvm-project@52c1d20

[NFC] Add more LLVM examples to references

[CMake] Update how include paths are set for `static`

Instead of relying on `INTERFACE_INCLUDE_DIRECTORIES` (set implicitly by
`target_include_directories`), call `target_include_directories` for
`static` and set the include path explicitly.

[NFC] Updte README.md

* Remove a note that no longer applies after the LLVM-10 update
* Update links to code (i.e. fix line numbers)

This patch also adds empty lines before lists and code blocks. This
change:
* makes my local Markdown viewer happier,
* doesn't affect how README.md is displayed on GitHub
* seems to be consistent with MarkdDown good practices

Switch from DenseMap to MapVector in StaticCallCounter

llvm::MapVector, as opposed to to llvm::DenseMap, guarantees that the
iteration order is identical to the insertion order. This simplifies the
implementation.

The tests for StaticCallCounter had to be updated as well. Although the
order of results is now deterministic, the results are no longer ordered
alphabetically when printing. Previously they were ordered to guarantee
determinism of the results.

README.md was updated accordingly.

[NFC] Update docs/comments for StaticCallCounter

This patch adds documentation and tests for the StaticCallCounter pass.
Some minor refactoring of the code was also done (to unify the style
across the codebase).

Simplify CMake for static

This patch refactors CMakeLists.txt for `static` so that it's more
straightforward and explicit (about it's dependencies).

[NFC] Unify how passes are registered

This simply updates how arguments are commented (a fix to adhere with
LLVM's coding style).

[CI] Clone only release/10.x branch and no history

When building LLVM from sources (in GitHub actions), only clone the
release/10x branch and ignore the history. This should speed up the
builds.

[CI] Run cron jobs more often

The GitHub actions limit for Open Source projects has been lifted, so
switching to more frequent builds.

[CI] Add GitHub actions for Windows

Builds and runs HelloWorld. LLVM is built from sources with Ninja as
generator.

[CI] Build LLVM on Ubuntu and Darwin every day

[NFC] Make sure that llvm-tutor builds without warnings

Item 1:
Some comments triggered -Wcomment as I used `\` at the end of a comment
line (which is interpreted as line-continuation). I've fixed that so
that the warning is no longer triggered.

Item 2:
LLVM's `setAlignment(unsigned)` has been deprecated:
 https://github.com/llvm/llvm-project/blob/release/10.x/llvm/include/llvm/IR/GlobalObject.h#L81
I've switched to `setAlignment(MaybeAlign)` instead.

Thank you @orestisf for drawing my attention to these!

I've also made some improvements to comments.

[NFC] Add a link to clang-tutor

[Docs] Explain the difference between analysis and transformation

Until now it wasn't really clear which pass falls into which category
and what the difference is. This has been inspired by my recent
discussion with @samolisov, thank you!

Add a link to the Obfuscator plugin from clang-tutor

cmake: specify 'SYSTEM' setting to disable warnings

https://cmake.org/cmake/help/latest/command/include_directories.html:
> If the SYSTEM option is given, the compiler will be told the
> directories are meant as system include directories on some platforms.
> Signalling this setting might achieve effects such as the compiler
> skipping warnings, or these fixed-install system files not being
> considered in dependency calculations - see compiler docs.

[NFC] Update instructions in README.md

As discussed in banach-space#12,
when loading libraries on Linux, one has to specify the directory. The
current working directory is not searched by dlopen.

Improve the documentation

* added missing `-analyze` when describing RIV
* refined the comparison between Analysis and Transformation passes

Add Dockerfile + Update README.md

* added Dockerfile and a Github Docker image workflow
* llvm-tutor uses clang to emit ll files from source files. When building LLVM from source we have to specify that we need clang to be built by setting LLVM_ENABLE_PROJECTS=clang cmake flag.

Add tests for existing LLVM passes

This patch adds tests for certain LLVM passes, which are relatively
straightforward, yet demonstrate some very interesting compiler
concepts. The tests help to understand/demonstrate the underlying pass.

[NFC] Small updated to README.md

Update section title

Add clarification regarding LLVM_DEBUG and STATISTIC

This is in relation to banach-space#20.

minor change to commands to fix warnings on linux (banach-space#17)

minor change to commands to fix warnings on linux

opt with -S produce .ll output and without that produce .bc output so
change commands accordingly where .ll file needed. Linux systems after
building llvm-tutor produce .so libraries while mac os produces .dylib
so change accordingly in commands for linux. Some places <source-dir> is
missing while specifying input files so fix them.

Add documentation

Fix banach-space#19

Fix Ratio::parser

The online documentation for parse method is incorrect:
  * https://llvm.org/docs/CommandLine.html#writing-a-custom-parser
The suggested signature is:
```
  bool parse(cl::Option &O, StringRef ArgName, const std::string &ArgValue,
             unsigned &Val);
```

However, from parser::parse in CommandLine.h:

```
bool parse(Option &O, StringRef ArgName, StringRef Arg, DataType &V)
```

Note that the 3rd argument should be a StringRef rather than
std::string.

Change "Linux 18.04" to "Ubuntu 18.04" in README.md (banach-space#21)

Switch llvm-tutor to LLVM 11

This patch updates llvm-tutor for the recently released LLVM 11. It
includes minor updates to the implementation of the passes, as well as
documentation and CI scripts.

Note that starting from LLVM 11 the requirement for CMake has changed.
LLVM (and also llvm-tutor) now requires CMake 3.13.4, which may mean
that some users will have to download CMake manually. For example,
Ubuntu 18.04 only offers 3.10.2. I hope that this won't be a big problem
- CMake is relatively easy to install: https://cmake.org/download/.

[NFC] Minor update in docs

Fix an issue with brew update/brew upgrade

I've only experienced this in GitHub Actions (hence only the GitHub
Actions files is updated). It manifested itself as follows:
```
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.
```

This patch implements a workaround from here:
* actions/runner-images#1811

Update references

Remove some old stuff. Added a great presentation by Jonathan Smith
that I haven't had the chance to watch until today.

Refine auto-registration for OpcodeCounter

OpcodeCounter is meant to be a reference example for how to register a
pass with one of the existing pipelines. However,
  * there were no tests for this
  * there was no standard registration for the new PM (i.e. via
  `-passes=opcode-counter`)
  * README.md was out-of-date

This patch updates the relevant code and adds a new test dedicated to
auto-registration.

[NFC] Update the CMake version required in the HelloWorld example

I forgot to update this when switching to LLVM 11. This is mostly for
consistency sake.

Update the LLVM version in the Windows CMake script

My CI on Windows has been failing for various reasons for the past few
months and I didn't notice this when updating llvm-tutor to LLVM 11.

Support for Windows remains "experimental".

Update msbuild GH action

This is required, as per
microsoft/setup-msbuild#33.

[NFC] Update GH action for Windows for testing

While I'm trying to fix building on Windows, I need this action to run
on every push.

Revert "[NFC] Update GH action for Windows for testing"

This reverts commit 0071fcd.

I've experimented a bit, but didn't get too far. Now the origina commit
is a nuisance when testing feature branches (builds on Windows and
fails as expected), hence reverting.

Add a printing pass, OpcodeCounterPrinter, for OpcodeCounter

I've learnt about the printing passes from this presentation by
Jonathan Smith (aka @jvs_tech):
  * https://www.youtube.com/watch?v=MagR2KY8MQI&t
It's a neat trick with which the behaviour of analyses passes is
consistent across pass managers (with respect to printing the results).

OpcodeCounter is the first pass in *llvm-tutor* that's updated to use a
printing pass. I've also updated the documentation as IMO this should be
the recommended way of writing such passes for the new PM. The pass
registration and tests are updated accordingly.

Fix the implementation of OpcodeCounterPrinter

This patch just fixes a silly typo. I'm a bit surprised that pretty much
everything just worked fine.

Add a printing pass for StaticCallCounter

The tests are updated accordingly. This patch also simplifies the
implementation of `static`.

I also took the liberty of renaming the tests. I think that now
filenames are more descriptive.

Add a printing pass for RIV

Switch from pip to pip3

Python 2 has reached end-of-life. This means that one should be using
pip3 instead of pip as well.

[GH] Remove the workaround for Darwin

I've been hitting various brew-related issues in GitHub Actions on
Darwin. I couldn't find a good fix, but I noticed that these were
triggered by the workarounds that I was using there. I removed the
workarounds and everything seems to work fine.

I may need to re-visit this, but for now I'm merging this.

[GA] Remove LT_OPT_PIPELINE_REG from build scripts

This CMake flag hasn't been supported for a while. I forgot to delete it
earlier.

[GA][NFC] Remove a call to `ls`

This call was originally added to aid debugging. It is no longer needed.

[NFC] Fix compiler warnings

A few non-functional changes and minor improvements guided by various
compiler warnings (extracted from the CI jobs). Adding in one patch as
their are really minor.

[GA] Add a workflow for statically registered passes

I'm currently implementing support for statically registered plugins on
a branch. However, I need this workflow in order to test that feature
branch. There seems to be no way of adding workflows on feature branches
only.

[GA] Switch to manual trigger for statically registered plugins

This workflow is currently for testing only. AFAIK, there's no way of
making it work with manual triggers before enabling it in some other way
first (e.g. trigger on _push_). Hence, I enabled it first so that it's
triggered on _push_. In this patch I switch to manual trigger.

[GA] Fix paths

[GA] Fix test path

[GA] Update how static registration is tested

I have now updated the feature branch, so this action needs to be
updated accordingly.

[CI] Add a manual trigger to one of the actions

I'm not entirely sure whether this will work, but I'm under the
impression that this needs to be updated on the main repository branch
before it can be tested.

[CMake] Add a workaround for banach-space#25

Because of the problem reported in banach-space#25, on Ubuntu Bionic we can't just
request LLVM 11.0.0 (or LLVM 11.0.1) with `find_package`. Instead, we
jsut add an extra check to make sure that LLVM's major version is indeed
11.

llvm-tutor will happily work with both LLVM 11.1.0 and LLVM 11.0.1.
However, on Darwin there's only LLVM 11.0.0 and on Ubuntu Bionic only
LLVM 11.1.0. So either we use:
  * find_package(LLVM 11.0.0 REQUIRED CONFIG) on Darwin
  * find_package(LLVM 11.1.0 REQUIRED CONFIG) on Ubuntu Bionic
or, we opt in for the solution implemented in this patch. Both are just
workarounds.

[CI] Make sure that we use LLVM 11.0.1 rather than 11.1.0

The release/11.x branch of llvm-project has recently been update and now
contains LLVM 11.1.0. Until recently it's been LLVM 11.0.0. This is
fine, but llvm-tutor should just stick to one version. In order to
guarantee this, use tags rather than branches to get a particular
version of LLVM.

I switched to `git clone --depth 10`. 10 is an arbitrary number here -
big enough to guarantee that the `llvmorg-11.0.1` tag is also
downloaded.

Add logic for static registration of plugins

In llvm-tutor we mostly use dynamic plugins. This patch adds
documentation, a bash script and a CI action to demonstrate static
plugin registration as an alternative to dynamic plugins.

The following presentation provides a great introduction into this
subject:

2020 LLVM Developers’ Meeting: S. Guelton
“Building Compiler Extension for LLVM 10.0.1”
https://www.youtube.com/watch?v=PkfTHdoUzT8&ab_channel=LLVM

The keep the script relatively simple, static_registration.sh bash will
only port the MBASub plugin.

I've also updated the GitHub action configuration that I've set-up for
testing this.

[CMake] Make sure that we use LLVM 11.0.1 rather than 11.1.0

This is similar to 99e499b, but
specifically for the Windows CMake script.

Disclaimer: Windows support is experimental.

Add include for CommandLine in StaticMain

[GA] Add triggers to workflows

Make x86-Darwin and x86-Ubuntu workflows run on PRs to master and on
manual triggers.

Add floating-point comparison analysis, printer, and transformation passes

[NFC] Update the docs and add some comments

Add necessary libraries to `static` executable

If you link against a dylib build (e.g. when building in debug mode)
you'll need these.

[GA] Update the CMake invocation to build with shared libs

[GA] Don't build shared libs for static plugins

With `BUILD_SHARED_LIBS=On` (set when building LLVM), static plugin
registration fails. I tested with:
  * LLVM_BYE_LINK_INTO_TOOLS set to On and Off
  * LLVM_MBASUB_LINK_INTO_TOOLS set to On and Off
This makes sense, `BUILD_SHARED_LIBS` and e.g.
`LLVM_BYE_LINK_INTO_TOOLS` are mutually exclusive (one is for shared
linking and the other for static). I should revisit this in the future
and perhaps prepare a patch for LLVM to make CMake catch this.

Add tests for FindFCmpEq + minor refactor

Add tests for FindFCmpEq to check that all comparisons that are _not
supported_ are indeed ignored by the pass.

I've also moved a few bits around to make the implementation more
consistent with other passes. These are NFCs.

[GA] Improve testing for static registration

Adds the following:
  * `-S` when running the sanity check so that the output in GitHub
  actions is human-readable (as opposed to being binary)
  * `-DLLVM_BUILD_EXAMPLES` so that the test for MBASub is indeed run
  (rather than being `UNSUPPORTED`)

[GA] Refine configs for actions that depend on pre-build packages

Added a cron trigger - this is to defend against unexpected changes to
the pre-build packages (e.g. unexpected switch from LLVM 11.0.0 to LLVM
11.1.0).

Update the branch for pull-request based triggers from `master` to
`main` (llvm-tutor switched to `main` a while back).

Update llvm-tutor to LLVM 12

Most notable changes:
1. The DIE pass was removed in https://reviews.llvm.org/D87933. DCE
   provides similar functionality and is used instead.

2. 'willreturn' was introduced in https://reviews.llvm.org/D62801. It
   means that the corresponding function is guaranteed to return (i.e.
   there are no infinite loops or calls to e.g. `abort` or `exit`). This
   means that it can be safely removed by the DCE pass. Without the
   `willreturn` attribute the DCE pass skips the corresponding function.

3. Multiple fixes in README.md that I noticed while updating it.

[Travis] Make sure that `lit` is run using Python 3 (instead of Python 2)

LLVM has recently switched to Python 3, i.e. Python 2 is no longer
supported. This also applies to `lit`, the Python script to runs LIT tests:
  * https://github.com/llvm/llvm-project/blob/release/12.x/llvm/utils/lit/lit.py
This change is required to make sure that in Travis CI environment,
`lit` is run using Python 3 rather than Python 2.

[NFC] Update CI links in README.md

[GA] Allow `workflow dispatch` for the docker configuration

Update the docker set-up

* Update the docker script from LLVM 11 to LLVM 12 (I missed it when
  upgrading other files).
* Switch the GitHub action file for Docker from tracking `master` to
  tracking `main` (again, missed it when updating the primary branch)

Add a docker file for Fedoro

llvm-tutor does not work in Fedora yet, but the docker file seems fine
and will be used for testing.

I've also renamed the Ubunty docker set-up/file. This is for
consistency.

Added check for /usr/lib64 for fix issue in Fedora 33

Add note about OptNone

The registration with the new pass manager requires that `OptNone` is not present for functions.
Make sure that clang-compiled BC files don't have OptNone

Switch to Fedora 34

IIUC,
* Fedora 32 comes with LLVM 10,
* Fedora 33 comes with LLVM 11, and
* Fedora 34 provides LLVM 12.

So we need Fedora 34 rather than 32.

I've also cleaned-up the docker file a bit.

Improve sanity checks in CMake

This is just a minor improvement to CMake after
banach-space@a139213.
I'm also adding some comments.

Fix Ubuntu actions

The Ubuntu GitHub actions started unexpectedly failing. Adding
`python3-setuptools` to the list of installed packages fixes the issue.

Update the documentation and implementation of `OpcodeCounter`

In banach-space#35, `-O1` was added
to the instructions for building `input_for_cc.bc` (i.e. the sample
input for the `OpcodeCounter` pass). Unfortunately, `-O1` causes all
instructions to be optimized away and hence there's nothing to print for
the `OpcodeCounter` pass. This patch fixes that by restoring the
original instruction for building `input_for_cc.bc` .

In order to work around `optnone` (the original rationale behind adding
`-O1` in the previous patch), `isRequired()` [1] is added to the
implementation of `OpcodeCounter.`

Fixes banach-space#38

[1] https://llvm.org/docs/WritingAnLLVMNewPMPass.html#required-passes

Fix banach-space#40

Background and solution identical to the previous patch.

Add -disable-output option for running FindFCmpEq

Add Dockerfile and GH workflow for Arch Linux

Apart from having a CI for yet another flavour of Linux, I wanted to see
whether I can reproduce the failure reported in banach-space#39. That's not the
case, so I suggest that we close banach-space#39.

Mod: Phi only have two incoming edges

Fix comment in DuplicateBB.cpp

Update the branch names in all links in README.md from master to main

Fix a typo in HelloWorld/CMakeLists.txt

Add "-c" arg in README.md and tools/StaticMain.cpp when expecting llvm bitcode output

If there isn't "-c" argument, clang will generate an error "clang-12: error: -emit-llvm cannot be used when linking".

[Docs] Add more detail on `optnone`

Fix a typo in lib/OpcodeCounter.cpp

Fix two typos in OpcodeCounter.cpp

Fix banach-space#50

 Add "Run the pass through opt - New PM" for InjectFuncCall

[GH Actions] Fix how LLVM 12 is installed

With LLVM 13 being released, `brew install llvm` means `brew install
llvm@13` rather than `brew install llvm@12`. And llvm-tutor is still
based on LLVM 12.

Bump llvm-tutor to LLVM 13

Quick summary of the changes:

1. ORC Jit has become the default JIT engine used in `lli`, see:
    * "[lli] Make -jit-kind=orc the default JIT engine"
    * https://reviews.llvm.org/D98931
    * https://releases.llvm.org/13.0.0/docs/ReleaseNotes.html#changes-to-the-llvm-tools
  However, in LLVM 13 it's a bit broken, see
    * https://bugs.llvm.org/show_bug.cgi?id=52030.
  This means that in tests that use `lli`, we need to specify
  `-jit-kind=mcjit` Once we switch to LLVM 14, we can delete remove this
  flag.

2. `-enable-new-pm` in `opt` now defaults to 1, which means that we need
   `-enable-new-pm=0` to make sure that the legacy pass manager is used
   instead of the new pass manager. Some tests only work with the legacy
   pass manager (e.g. the ones that use call-site alwaysinline). Tests
   are updated accordingly.

Fix Darwin workflow

Switch away from the deprecated `CreateLoad`

Fix banach-space#53

Update Fedora Dockerfile to fix build

Latest Fedora 35 release has LLVM 13 package

Add an Ubuntu Docker configuration

This patch adds a Docker file for building llvm-tutor on Ubuntu using
the pre-build binary packages from https://apt.llvm.org/. A GitHub
action is also added to run this configuration.

I'm adding this as this configuration seems to be broken and I need an
easy way to reproduce it. I suspect that the Ubuntu packages might be
missing some dependency.

Fix Ubuntu configuration

The `apt` packages for Ubuntu have recently been updated to include
MLIR. This has triggered an update in the LLVM CMake modules and
`libmlir-13` is now required to be installed alongside other LLVM
packaged (otherwise CMake configuration fails). I've added a couple more
LLVM packages (`libmlir-13-dev` and `libllvm13`), just to be on the safe
side.

To make things a bit easier for myself, I've switched from Ubuntu 18.04
to 20.04. I think that this is rather safe given that 20.04 is almost 2
years old.

Fix GA file (use `{opt|clang}-13` instead of `{clang|opt}`)

Fix Ubuntu docker file (incorrect LLVM installation dir)

Updat Travis links

[CI] Update the Travis config

Update the Ubuntu dependencies

Fixed typo

Simplify a heredoc in static_registration.sh

Replace `git://` with `https://`

See
https://github.blog/2021-09-01-improving-git-protocol-security-github/
for context.

Make HelloWorld pass required

Fix docs for running lli in LLVM 13 and add an end-to-end test for DynamicCallCounter (banach-space#65)

* Fix docs for running lli in LLVM 13

The default JIT engine, ORC JIT, used by lli in LLVM 13 is broken. This commit updates the docs for running lli using MCJIT, reflecting changes already made in the tests.

* Create an end-to-end test for DynamicCallCounter

An end-to-end test consumes a C file as input, so that it verifies invocations of clang, opt, and lli.

* Update commands for DynamicCallCounter

Make two updates on the commands for running DynamicCallCounter:
- Remove redundant parameters for invoking clang.
- Fix the invocation of opt for legacy PM.

Extend GitHub actions

Make the basic GitHub actions use both GCC and Clang, and to build both
Release and Debug.

just a couple of typo fixes!

add enable-new-pm=0 to enable legacy pass manager

Add a note about the LLVM 14 switch

Refine find_package

Update to LLVM 14

Bump Fedora version

Remove scripts for Windows

I've not had the time to resurrect the Windows support, so instead I am
removing all Windows specific files that are left. Otherwise, folks
might assume that Windows is supported (unfortunately, it is not).

[CI] Bump Ubuntu version to 20.04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Apple Area: Packages investigate Collect additional information, like space on disk, other tool incompatibilities etc.
Projects
None yet
Development

No branches or pull requests

15 participants